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ABSTRACT 


* In  only  the  last  25  years  has  operator  reliability  been  incorporated 
with  hardware  reliability  to  obtain  a value  for  the  effectiveness  of  a man- 
machine  system.  Very  seldom,  and  in  most  cases,  never,  do  any  of  the  human 
reliability  models  address  the  effect  that  operator  learning  has  on  human 
reliability  and  the  subsequent  impact  that  operator  reliability  has  on 
system  effectiveness.  This  research  studied  the  sensitivity  of  a system 
effectiveness  model  to  changes  in  operator  learning  levels. 

Learning  data  which  was  expressed  in  terms  of  performance  versus  time, 
was  obtained  from  a paper  which  analyzed  the  performance  of  an  actual  manu- 
facturing task.  This  data  was  utilized  to  develop  three  different  curves  - 
a log  pseudo-learning  curve,  a cubic  pseudo-learning  curve,  and  a Learn- 
Forget-Learn  (LFL)  curve.  Each  curve  expressed  operator  performance  as  a 
function  of  time. 

The  expressions  for  each  cf  the  three  curves  were  then  utilized  in 
conjunction  with  a system  effectiveness  simulation  model  to  formulate 
values  for  system  effectiveness.  The  various  values  of  system  effectiveness 
obtained  from  the  simulation  demonstrated  that  the  model  was  sensitive  to 
changing  levels  of  operator  performance. 

This  research  is  unique  because  this  is  the  first  time  that  operator 
learning  curves  have  been  utilized  in  conjunction  with  a simulation  of 


system  effectiveness. 
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1 .1  Problem  Statement 


In  a 1960  survey  of  nine  Air  Force  missile  systems,  it  was  found  that 
human  error  contributed  from  20  to  53  percent  to  system  unreliability  (26). 
Another  study  (24)  investigated  a large  number  of  production  defects  at  the 
plant  of  a prime  contractor  for  the  Atomic  Energy  Commission  and  it  was 
determined  that  82  percent  of  defects  found  by  inspectors  could  be  directly 
attributed  to  human  error.  The  above  figures  demonstrate  that  human  per- 
formance has  a significant  impact  on  the  reliability  of  a system.  Because 
of  this,  a concentrated  effort  has  been  made  in  the  past  25  years  to  combine 
human  reliability  values  with  hardware  reliability  figures  to  obtain  an 
overall  reliability  index  for  the  man-machine  system. 

Numerous  human  reliability  models  have  been  formulated  that  attempt 
to  assign  reliability  values  to  an  operator's  performance  in  a man-machine 
system.  Some  of  these  models  attempt  to  demonstrate  that  relationships 
exist  between  the  level  of  operator  performance  and  factors  such  as:  amount 
of  supervision,  working  environment,  willingness  to  work,  etc.  Very  seldom, 
and  in  most  cases  never,  do  any  of  these  human  reliability  models  address 

the  impact  of  an  operator's  rate  of  learning  on  the  human  performance  level 

■ 

and  subsequently  on  the  reliability  of  the  system. 

The  problem  that  will  be  addressed  by  this  study  is  the  impact  that 

(changing  rates  of  learning  have  on  an  operator's  performance  level.  This 
change  in  an  operator's  performance  for  various  time  increments  will  be 
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found  by  utilizing  a Learn-Forget-Ldarn  (LFL)  curve  and  two  learning  curves. 
The  learning  curves  and  LFL  curve  will  be  developed  using  data  from  a previous 
study  of  learning.  A simulation  model,  developed  by  another  researcher  (16), 
which  quantifies  System  Effectiveness  (or  reliability)  will  be  used  in  con- 
junction with  the  results  from  the  two  curve  types  (learning  & LFL)  to  study 
the  impact  that  learning  has  on  the  system  reliability  index.  The  sensi- 
tivity of  the  system  reliability  index  to  changing  rates  of  learning  will  be 
calculated,  analyzed  and  discussed.  It  will  be  shown  that  operator  learning 
does  have  an  effect  on  the  total  reliability  value  of  the  system. 

1 .2  Purpose 

The  main  purpose  of  this  study  is  to  demonstrate  that  operator  learning 
for  various  increments  of  time  will  have  a significant  impact  on  operator 
performance  and  also,  therefore,  on  System  Effectiveness.  The  secondary 
purpose  is  to  outline  the  requirement  for  additional  and  more  thorough 
research  in  the  area  of  operator  learning  and  its  subsequent  impact  on  the 
reliability  of  the  overall  man-machine  system. 

The  study  is  organized  so  as  to  lend  support  to  the  main  theme  that 
was  outlined  above.  The  literature  survey,  which  follows  this  section, 
reviews  a number  of  human  reliability  models  that  have  been  proposed. 

Only  two  of  these  models  address  the  aspect  of  operator  learning/training 
to  any  great  detail.  The  remaining  models  make  no  reference  to  operator 
training/learning  and  their  impact  on  the  level  of  human  performance.  An 
in  depth  comparison  of  various  models  will  be  presented  in  a table  that, 
will  outline  the  positive  and  negative  aspects  of  the  models.  The  liter- 
ature survey  also  contains  a review  of  articles  that  pertain  to  learning 
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curves.  The  uses  and  methodology  of  learning  curves  will  be  presented 
along  with  examples  of  typical  curves.  The  graphical  and  mathematical 
representation  of  the  learning  curves  will  also  be  discussed.  The  liter- 
ature survey  also  will  outline  the  methodology  of  LFL  curves,  but  only  one 
reference  was  discovered  which  attempted  to  explain  and  discuss  this  type 
of  curve.  The  techniques  and  conclusions  of  this  reference  will  be  out- 
lined in  the  literature  survey. 

Chapter  2 of  this  report  is  devoted  to  the  development  of  two  learning 
curves  and  an  LFL  curve.  Data  obtained  from  one  of  the  references  on 
learning  curves  will  be  used  as  a basis  to  plot  two  different  learning 
curves.  The  LFL  curve  will  be  plotted  using  the  data  presented  in  the 
lone  LFL  curve  reference.  The  characteristics  of  each  curve  will  be  ex- 
plained and  mathematical  expressions  for  the  three  curves  will  be  developed. 

Chapter  3 presents  a summary  and  explanation  of  a System  Effectiveness 
model  which  has  been  developed  by  another  researcher  (16).  The  "personnel" 
term  associated  with  the  equation  for  this  model  will  be  analyzed  in  more 
detail,  especially  in  relation  to  learning.  The  effects  on  this  "personnel" 
or,  preferably,  "operator"  term  for  changing  rates  of  learning,  obtained 
from  the  respective  curves,  will  be  analyzed  in  terms  of  operator  performance. 
The  sensitivity  of  the  System  Effectiveness  model  to  these  fluctuating  levels 
of  operator  performance  then  will  be  studied  and  discussed.  Conclusions  then 
will  be  formulated  concerning  the  impact  of  operator  learning  on  the  System 
Effectiveness  model. 

The  last  section  of  the  study  will  outline  the  requirement  for  additional 
research  in  the  area  of  operator  learning  and  its  effect  on  the  reliability 
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of  a man-machine  system.  The  question  of  what  impact  does  operator 
learning  have  on  operator  performance  is  of  critical  importance  to  system 
rel iabil ity. 

1 .3  Literature  Survey 

This  study's  three  main  areas  of  interest  (human  reliability  models, 
learning  curves  and  LFL  curves)  will  be  addressed  separately  in  this 
survey  of  the  literature.  There  are  abundant  references  in  the  literature 
which  pertain  to  the  prediction  of  human  performance  in  man-machine  systems. 
A summary  of  a representative  and  well  known  method  for  predicting  human 
performance,  THERP,  will  be  presented.  There  are  also  numerous  articles 
devoted  to  learning  curves,  but  references  related  to  Learn-Forget-Learn 
(LFL)  curves  are  very  few  in  number,  almost  to  the  point  of  being  non- 
existant. 

1.3.1  Human  Reliability  Models 

1.3. 1.1  Early  Studies 

In  the  past,  reliability  figures  were  calculated  for  a man-machine 
system  based  solely  on  the  machine  component  of  the  system.  The  human 
component  was  assumed  to  be  totally  reliable  and  no  provisions  were  in- 
cluded in  the  models  to  account  for  human  unreliability.  Later  on  it  was 
determined  that  the  human  aspects  of  man-machine  systems  contributed  greatly 
to  the  system  unreliability,  in  some  instances  even  more  so  than  the  equip- 
ment component  [27].  After  this  discovery,  much  more  emphasis  was  placed 
on  predicting  human  performance  in  a system. 
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One  of  the  earliest  man-machine  reliability  studies  in  which  human 
error  rates  were  estimated  and  related  to  estimates  of  equipment  malfunction 
rates  was  done  in  1952  by  an  electronics  engineer  and  a mathematician  at 
Sandia  Corporation  (27).  The  treatment  of  human  error  in  this  1952  study  of 
an  aircraft  nuclear  weapon  system  was  crude.  Only  those  errors  which 
would  directly  reduce  system  reliability  without  any  other  equipment  failure 
or  human  error  being  involved  were  studied.  The  estimates  of  human  error 
were  included  in  the  overall  system  reliability  equation  and  were  treated  in 
the  same  manner  as  estimates  of  failures  rates  for  other  system  factors. 

Later  studies  became  more  refined  in  regards  to  the  quantitative 
methods  utilized  for  evaluating  human  performance  and  its  relationship  to 
man-machine  system  performance.  One  report  recommended  a)  making  rough 
estimates  of  the  probability  of  successful  completion  of  each  sub-task  in 
a system  and  then  b)  combining  the  probabilities  to  obtain  the  overall 
reliability  of  the  system  (32).  Another  researcher  pointed  out  that  it  was 
necessary  to  treat  those  rough  error  rate  estimates,  mentioned  in  the  above 
study,  differentially  according  to  their  importance  to  system  performance  (23). 
He  defined  task  criticality  in  quantitative  terms  related  to  the  effect  of 
unsuccessful  task  completion  upon  system  success.  Eventually,  more 
sophisticated  models  were  developed  to  predict  human  performance  reliability 
more  accurately.  One  of  these  methods  was  called  Technique  for  Human  Error 
Rate  Prediction  (THERP). 


1.3. 1.2  Technique  for  Human  Error  Rate  Prediction  (THERP). 

THERP  is  one  of  the  best  known  methods  developed  to  quantify  human 
performance.  In  1961,  Swain  (28)  developed  this  method  for  evaluating  the 
human  error  contribution  to  system  degradation.  The  following  discussion 
of  THERP  is  almost  entirely  from  a paper  entitled  "Methods  of  Predicting 
Human  Reliability  in  Man-Machine  Systems"  by  David  Meister  of  the  Bunker- 
Ramo  Corporation  (17). 

THERP  has  been  used  primarily  to  provide  quantitative  predictions  of 
system  degradation  resulting  from  human  errors  in  association  with  equipment 
reliability,  operational  procedures,  and  other  system  characteristics  which 
influence  human  behavior.  THERP  is  an  iterative  procedure  that  consists  of 
five  steps  which  are  repeated,  not  always  in  the  same  order,  until  system 
degradation  resulting  from  human  error  is  at  an  acceptable  level.  The 
five  steps  are  listed  below. 

"(1)  Define  the  system  or  subsystem  failure  which  is  to  be 
evaluated. 

(2)  Identify  and  list  all  the  human  operations  performed  and  their 
relationships  to  system  tasks  and  functions. 

(3)  Predict  error  rates  for  each  human  operation  or  group  of 
operations  pertinent  to  the  evaluation. 

(4)  Determine  the  effect  on  human  errors  on  the  system. 

(5)  Recommend  changes  as  necessary  to  reduce  the  system  or  subsystem 
failure  rate  as  a consequence  of  the  estimated  effects  on  the 
recommended  changes . " 

I I 
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Swain  (29)  points  out  that  "the  steps  are  typical  of  the  usual  system 
reliability  study  if  one  suDstitutes  'hardware1  for  'human'." 

The  goals  of  this  technique  are  listed  by  Meister  in  another  report 
entitled  "Comparative  Analysis  of  Human  Reliability  Models"  (18).  They 
are: 

"(1)  To  derive  'quantitative  estimates  of  the  degradation  to  a 
man-machine  system  resulting  from  human  error.' 

(2)  Or,  'to  evaluate  the  human  error  contribution  to  systems 
degradation. ' 

(3)  To  predict  human  error  rates. 

(4)  To  determine  those  design  changes  to  the  system  necessitated 
by  the  system  failure  rate." 

One  of  the  assumptions  associated  with  THERP,  as  listed  in  (18)  by 
Meister,  is: 

"THERP  takes  into  account  various  psychological  and  physiological 
stresses,  training,  motivation  and  situational  factors.  These 
are  called  Performance  Shaping  Factors  (PSF)  and  they  are  very 
subjective  in  their  application." 

In  regard  to  the  above  assumption,  Meister  makes  the  following 
comment: 

"These  factors,  i.e.  PSF,  must  be  taken  into  account  in  the  gathering 
of  error  rate  data  and  the  error  estimates  derived  should  be  modified 
in  accordance  with  the  presumed  effect  of  these  factors  on  performance. 
One  difficulty  that  arises,  however,  in  accounting  for  these  molar 
factors  on  performance  is  the  difficulty  of  recognizing  their  influence 
and  estimating  the  extent  of  that  influence." 

This  statement  by  Meister  embodies  the  purpose  of  this  study,  i.e.  to 
recognize  the  influence  and  estimate  the  extent  of  that  influence  on  human 


performance  caused  by  the  so-called  Performance  Shaping  Factors,  specifically, 
the  factor  of  learning.  THERP  has  no  specific  provisions  to  handle  varying 
levels  of  operator  learning  nor  to  predict  the  impact  that  these  learning 
levels  have  on  human  performance.  This  study  will  hopefully  demonstrate  the 
effect  that  learning  has  on  operator  performance. 

1.3. 1.3  Comparison  of  Human  Reliability  Models 

Numerous  human  reliability  models  have  been  formulated  that  attempt 
to  predict  operator  performance  levels  in  man-machine  systems.  Davis  Meister, 
in  the  report,  "Comparative  Analysis  of  Human  Reliability  Models,"  summarized 
and  characterized  18  human  reliability  models  (18).  Table  1.1  is  an 
abbreviated  version  of  Meister's  "Summary  of  Model  Characteristics"  which 
can  be  found  on  page  414  of  (18).  It  should  be  noted  that  for  the  table's 
sub-category  of  "Selection/Training"  only  two  models,  the  Human  Operator 
Simulator  (HOS)  and  the  Personnel  Reliability  Index,  meet  the  criteria 
established  for  that  sub-category  by  Meister.  For  a complete  description 
of  all  criteria  used  in  the  table,  consult  pages  413  through  425  of  (18). 

Meister  makes  the  following  remarks  concerning  the  sub-category  of 
"Selection/Training": 

"Most  of  the  methods  possess  little  or  no  capability  in  the  areas  of 
manpower  selection  and  training  despite  the  fact  that  claims  for 
these  capabilities  are  often  made.  We  feel  that  to  be  sensitive  to 
training,  a model  must  Indicate  what  capabilities  should  be  trained, 
rather  than  merely  that  additional  training  is  required.  On  that 
basis  only  a few  of  the  models,  i.e.,  the  personnel  reliability 
technique  of  Siegel  and  Wherry's  HOS,  seem  to  possess  this  sensitivity. 

It  may  be  that  the  majority  of  the  models  available  do  not  include 
parameters  which  are  sensitive  to  the  factor  of  training  or  it  may 
be  that  a distinctly  different  type  of  model  Is  required. 

This  comparison  of  human  reliability  models  points  out  that  very  little 

work  has  been  done  in  the  area  of  training  and  learning  with  their  attendant 
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COMPARISON  OF  HUMAN  RELIABILITY  MODELS 


Maintainability  Prediction 


impact  on  human  performance.  This  study  will  demonstrate  that  operator 
learning  has  a significant  effect  on  human  performance. 

1.3.2  Learning  Curves 

In  1936  Wright  (33)  published  the  first  article  that  formulated  the 

theory  of  learning  curves.  He  noted  a continuous  improvement  in  labor 

cost  in  the  manufacture  of  airplanes  as  the  workers  repeated  their  tasks. 

From  his  observations  and  study  in  the  aircraft  industry,  he  developed 

the  basic  learning  curve  theorem  which  can  be  stated  as: 

"For  any  operation  which  is  repeated,  the  time  of  the  operation 
will  decrease  by  a fixed  fraction,  known  as  the  reduction  fraction, 
each  time  the  number  of  operations  doubles." 

Learning  curves  are  applicable  to  many  aspects  of  production  planning 
and  control.  They  can  be  used  to  predict  the  cost  per  unit  of  production, 
offer  quantity  discounts,  and  establish  selling  price.  Learning  curves 
also  influence  delivery  schedules,  set  labor  standards,  and  measure  shop 
efficiency  (2).  They  can  also  be  utilized  for  establishing  costs  of  manu- 
facture  and  determining  labor  requirements. 

The  learning  curve  is  actually  a line  on  a graph  which  demonstrates 
the  reduction  of  time  in  any  repetitive  operation.  Two  facts  concerning  the 
use  of  learning  curves  are  important:  (1)  The  time  required  to  do  a job 
will  decrease  each  time  the  job  is  repeated.  (2)  The  amount  of  decrease 
will  be  less  with  each  successive  unit. 

The  curve  may  be  presented  on  any  type  graph  paper  but  is  more  commonly 
portrayed  on  log-log  graph  paper.  When  plotted  on  arithmetic  graph  paper, 
the  shape  of  a typical  learning  will  be  exponential  as  demonstrated  in 


n 

Figure  1.1.  It  can  be  noted  that  the  curve  possesses  the  characteristics 
of  an  initial  rapid  fall  followed  by  a flattening  of  the  curve  and  after  a 
relative  small  number  of  repetitions,  the  rate  of  improvement  is  small. 

If  log-log  graph  paper  is  used  instead  of  arithmetic  graph  paper,  a 
straight  line  is  presented.  Figure  1.2  is  identical  with  Figure  1.1  except 
that  the  points  have  been  plotted  on  log-log  paper.  The  nature  of  log-log 
scales  permits  the  inclusion  of  many  repetitions  or  long  periods  of  time 
which  would  be  impossible  with  arithmetic  graph  paper.  On  log-log  paper, 
the  distance  between  doubled  quantities  is  equal.  This  fact  coupled  with 
the  learning  curve  theorem  is  why  the  plot  of  a learning  curve  on  log-log 
paper  is  linear. 

The  learning  curve  is  a power  curve  of  the  form: 

+ _ + -m 
tn  ‘ tln 

where: 

tn  * the  time  of  operation  number  n 
t^  = the  time  of  the  first  operation 
n * number  of  repetitions 
m * slope  of  the  curve 

When  reduced  to  logarithmic  form,  this  equation  is  represented  by  the 
linear  equation:  log  tn  = log  t^  - m log  n.  The  slope  of  the  line,  m, 
is  frequently  called  the  reduction  fraction  and  it  represents  the  rate 
of  learning.  The  reduction  fraction  usually  varies  between  .7  and  .95 
depending  on  the  proportion  of  labor  in  the  task  which  is  man-controlled  (8). 
The  complexity  of  the  task  and  human  motivation  are  also  factors  which 
affect  the  reduction  fraction. 


Operation  Time  (hours) 


Operation  Time  (hours) 


FIGURE  1.2 

EXPONENTIAL  LEARNING  CURVE  PLOTTED 
ON  LOG-LOG  COORDINATES 
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Numerous  other  equations,  more  complex  and  involved  than  the  above 
equation  have  been  developed  to  express  the  learning  curve  theory  (31), 

(5)  and  (15).  One  of  the  equations  fits  an  S-curve  to  learning  phenomenon 
while  another  gives  an  expression  for  a more  complex  exponential  curve. 

Carlson  and  Rowe  (6)  advocate  that  a learning  curve,  plotted  on 
arithmetic  graph  paper,  will  have  an  S-shape  instead  of  the  exponential 
shape  proposed  by  Wright  (33).  See  Figure  1.3  for  an  example  of  the 
S-shaped  curve.  They  maintain  that  the  "incipient"  phase  generally  in- 
volves little  improvement  because  the  worker  is  getting  accustomed  to  the 
shop  setup,  tooling,  instructions,  workplace  arrangement,  and  the  conditions 
of  the  process.  The  second  phase,  "learning,"  is  where  most  of  the  im- 
provement takes  place  because  this  phase  includes  the  reduction  in  errors, 
development  of  a work  pattern,  and  rearrangement  of  the  workplace.  The 
third  phase,  "maturity,"  represents  a limit  to  improvement  because  some 
learning  still  takes  place  but  at  a much  slower  rate  and  becomes  asymptotic  to 
the  limit. 

Numerous  discussions  have  taken  place  concerning  the  advantages  and 
disadvantages  of  Wright's  simple  equation  compared  to  the  more  complex 
expressions.  It  has  been  pointed  out  that  deficiencies  exist  in  the 
practical  use  of  the  power  form  model  of  Wright  (5).  Two  of  these  defici- 
encies are  the  model 's  ultimate  asymptote  of  zero  and  the  infinite  learning 
period  i.e.  learning  rate  is  assumed  to  be  constant.  The  advantages  of 
Wright's  equation  are  its  simplicity  and  ease  of  calculations.  Also,  it  is 
more  easily  understood  by  management  than  the  more  complex  models.  Even 
though  the  disadvantages  of  Wright's  equation  are  significant,  the  consensus 
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has  been  that  the  simple  straight  line  on  log-log  paper  is  best  (11). 

Corlett  and  Morcombe  (8)  state  that  there  are  not  many  industrial 
studies  reported  of  the  use  of  learning  curves  in  the  field  of  training, 
but  it  is  in  training,  where  learning  is  taking  place  continually,  that  it 
should  have  the  most  applications.  This  study  will  apply  two  different 
learning  curve  equations  to  ascertain  the  performance  level  of  an  operator 
over  various  periods  of  time.  These  various  performance  levels  will  then 
be  utilized  in  an  equation  for  System  Effectiveness,  and  the  sensitivity 
of  the  System  Effectiveness  index  to  the  changing  performance  levels  will 
be  studied. 

1.3.3  LFL  Curves 

Very  little  research  has  been  done  in  the  area  of  Learn-Forget-Learn 
(LFL)  curves  and  their  impact  on  human  performance.  LFL  curves  usually 
have  a saw-tooth  shape  as  can  be  seen  in  Figure  1.4.  This  shape  is  the 
result  of  an  operator  learning  a particular  task  for  a certain  time  period 
and  then  having  that  learning  interrupted  by  some  event  which  takes  him 
away  from  the  task.  In  all  probability,  he  will  forget  a portion  of  what 
he  had  originally  learned,  and  his  performance  on  the  original  task  will 
decline.  This  sequence  of  events  account  for  the  curve  shape,  i.e.,  the 
initial  learning  is  depicted  as  a gradual  increase  in  the  curve  followed 
by  a more  pronounced  increase,  but  when  the  interruption  of  learning  occurs, 
the  curve  drops  off  and  operator  performance  decreases.  When  the  operator 
returns  to  the  task  after  the  interruption,  his  performance  starts  to  in- 
crease again  as  the  curve  begins  to  climb.  The  learning/ forgetting  curve 
explained  above  has  been  proposed  by  Carlson  and  Rowe  (6). 
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It  should  be  noted  that  the  forgetting  portion  of  Figure  1.4  shows  a 
rapid  initial  decrease  in  performance  followed  by  a gradual  leveling  off 
as  a function  of  the  interruption  interval  period.  Also,  the  rate  and 
amount  of  forgetting  decreases  as  an  increased  number  of  units  are  com- 
pleted before  an  interruption  occurs.  These  two  attributes  of  forgetting 
curves  demonstrate  that  the  amount  of  forgetting  and  the  corresponding  level 
of  performance  are  a function  of  both  the  performance  at  the  time  the  task 
was  interrupted  and  the  length  of  the  interruption. 

From  the  above  discussion,  it  can  be  deduced  that  an  LFL  is  a combin- 
ation of  learning  and  forgetting  curves  over  various  periods  of  time. 

The  LFL  curve  will  be  explained  in  more  detail  in  the  following  chapter  to 
include  graphical  and  mathematical  derivations  of  the  curve. 
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CHAPTER  l 

DEVELOPMENT  OF  CURVES 

2.1  Introduction 

Three  different  curves  will  be  developed  in  this  chapter.  Two  of  the 
curves  will  be  pseudo-learning  curves  while  the  third  will  be  a LFL 
curve.  The  learning  curves  are  referred  to  as  pseudo-learning  curves  because 
of  the  coordinates  used  to  plot  the  curves.  A normal  learning  curve  is 
usually  plotted  using  "Cumulative  Units"  as  the  independent  variable  and 
"Time  per  Unit"  as  the  dependent  variable,  but,  for  the  purposes  of  this 
paper,  "Calendar  Weeks"  will  be  utilized  as  tne  independent  variable  and 
"Performance"  as  the  dependent  variable.  The  reasoning  behind  this  change 
in  coordinates  is  to  insure  that  the  units  of  the  results  obtained  from  the 
learning  curves  will  be  compatible  with  the  units  utilized  in  the  System 
Effectiveness  model  because,  in  Chapter  3,  it  is  required  to  have  time 
as  the  independent  variable,  and  reliability  is  expressed  over  time.  For 
the  purposes  of  this  paper,  the  units  of  the  dependent  variable,  "Performance", 
will  be  defined  in  terms  of  probability  of  success,  i.e.  reliability.  For 
example,  a performance  value  of  35  percent  implies  35  hits  out  of  100 
attempts  for  an  infantryman  shooting  at  a target.  It  could  also  imply  35 
correct  observations  out  of  100  total  observations  for  a radar  or  sonar 
operator.  The  above  definition  of  performance  will  be  explained  in  more 
detail  in  the  following  sections  of  this  chapter. 

The  first  learning  curve  is  expressed  by  a log  equation  which  is 
similar  to  the  equation  of  the  first  learning  curve  proposed  by  Wright  (33). 

The  second  learning  curve  has  been  formulated  in  terms  of  a cubic  equation. 
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The  LFL  curve  is  expressed  by  three  different  equations  depending  on  the 
section  of  the  curve  under  study.  Data  used  to  develop  the  three  curves, 
(Tables  2.1,  2.3,  and  2.5),  was  obtained  from  Carlson  and  Rowe  (6). 

Carlson  and  Rowe  accumulated  this  data  by  studying  the  performance  of 

60  individuals  who  performed  the  same  skilled  manual  tasks  in  a manufacturing 

plant. 

The  data  presented  by  Carlson  and  Rowe  (6)  was  expressed  in  terms  of 
calendar  weeks  versus  performance  where  performance  was  defined  as  the 
ratio  of  standard  time  to  actual  time.  For  the  purposes  of  this  study, 
performance  is  redefined  so  as  to  express  probability  of  success  or  reli- 
ability. Because  of  this  new  definition,  performance  means  probability  of 
success  and  not  the  usual  measure  of  quantity  output,  thus  the  value  of 
performance  must  be  less  than  or  equal  to  100%.  Because  of  this  constraint 
on  the  values  of  performance,  the  data  obtained  from  (6)  had  to  be  normalized 
because  some  of  the  performance  values  were  in  excess  of  100%.  This  trans- 
formation of  the  performance  variable  is  required  so  that  probability  of 
success  is  expressed  over  time.  This  requirement  will  become  evident  in 
Chapter  3.  Therefore,  the  basic  hypothesis  behind  the  redefinition  of  the 
performance  variable  is  that  the  probability  of  success  is  a one-to-one 
transformation  with  the  observed  performance  data,  that  is,  it  was  assumed 
to  have  the  same  form.  Hence,  the  observed  performance  data,  as  presented 
in  (6),  was  utilized  to  generate  the  probability  of  success  data  which  was 
used  to  develop  the  learning  curves. 

The  data  for  the  log  pseudo-learning  curve  and  the  LFL  curve  was  obtained 
by  normalizing  the  original  data  presented  in  (6)  so  that  no  performance 
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values  were  in  excess  of  100  percent.  All  values  were  normalized  because 
of  the  definition  of  "Performance"  i.e.  probability  of  success  (reliability) 
can  not  exceed  a value  of  1.00  which  is  the  same  as  a performance  value  of 
100  percent.  The  original  data  was  utilized  to  plot  the  cubic  pseudo- 
learning curve  because  there  were  no  performance  values  which  exceeded  100 
percent. 

The  results  from  the  three  different  curves  will  be  utilized  in  Chapter  3 
in  conjunction  with  a model  that  formulates  System  Effectiveness.  The 
sensitivity  of  this  System  Effectiveness  model  to  the  different  curve  types 
and  to  changing  performance  values  associated  with  the  curves  will  be 
analyzed. 

2.1.1  Log  Pseudo-Learning  Curve 

This  learning  curve  is  very  similar  to  the  log-linear  learning  curve 
developed  by  Wright  in  1936  (33).  The  curve  developed  by  Wright  is  the 
simplest  and  most  easily  understood  of  all  the  learning  curves  which 
have  been  developed.  Figure  1.1  is  an  example  of  Wright's  curve  when  it 
is  plotted  using  Cartesian  coordinates.  Its  simplicity  and  ease  of  cal- 
culation make  it  the  most  widely  used  learning  curve. 

The  data  in  Table  2.1  has  been  used  to  plot  the  log  pseudo-learning 
curve  of  Figure  2.1.  Again,  this  data  is  the  result  of  normalizing  the 
original  data  presented  by  Carlson  and  Rowe  (6). 

The  model  for  the  log  pseudo-learning  curve  was  developed  using  the 
linear  regression  program  of  the  Statistical  Analysis  System  (SAS).  The 
model  has  the  following  form:  P(t)  = 31.534  + 19.549  log  t 
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where:  P(t)  = Performance,  percent 
t = Time,  calendar  weeks 

The  correlation  coefficient  for  this  model  is  r = .997.  Table  2.2  is  the 
table  of  residuals  (residual  = observed  value  - predicted  value)  of  the 
model  compared  to  the  observed  values  (Table  2.1).  It  can  be  noted  in 
Table  2.2  that  the  model  gives  an  extraordinarily  good  fit  to  the  observed 
data. 

In  practical  terms,  this  type  of  learning  curve  would  result  from  a 
work  situation  where  the  individual  works  continuously  on  the  same  job, 
i.e.  he  is  not  detailed  or  assigned  to  tasks  other  than  his  main  job 
assignment.  An  example  of  this  type  of  situation  would  be  a radar  operator 
who  does  nothing  else  except  monitor  the  radar  screen.  If  the  individual  is 
interrupted  while  working  at  his  primary  job  assignment,  this  type  of 
learning  curve  would  not  be  applicable.  Section  2.1.3  addresses  this  type 
of  interrupted  learning  experience. 

2.1.2  Cubic  Pseudo-Learning  Curve 

The  cubic  learning  curve  was  proposed  some  years  after  Wright's  log- 
linear  formulation  (33).  It  was  developed  in  an  effort  to  eliminate  the 
two  major  disadvantages  of  the  log-linear  form  i.e.  the  zero  asymptote  and 
the  assumed  constant  rate  of  learning.  This  learning  curve  plots  on 
Cartesian  coordinates  as  an  S-shaped  curve  when  "Cumulative  Units"  is  used 
as  the  independent  variable  and  "Performance"  is  used  as  the  dependent 
variable.  See  Figure  1.3  of  Chapter  1.  Many  references  exist  in  the  liter- 
ature which  address  the  theory  and  formulation  of  cubic  curves  (1),  (6) 
and  (7). 
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FIGURE  2.1 

LOG-PSEUJO  LEARNING  CURVE 


Model:  P(t)  = 31.534  + 19.549  log  t 


Observed  Predicted 


Week 

Performance  Value 

Performance  Value 

Residual 

1 

34.400 

31.534 

2.866 

2 

47.000 

45.084 

1.916 

3 

53.600 

53.011 

0.589 

4 

58.300 

58.634 

-0.334 

5 

62.100 

62.997 

-0.897 

6 

65.400 

66.561 

-1.161 

7 

68.300 

69.574 

-1.274 

8 

70.800 

72.185 

-1.385 

9 

73.000 

74.487 

-1.487 

10 

75.100 

76.547 

-1 .447 

11 

77.000 

78.410 

-1.410 

12 

78.800 

80.111 

-1.311 

13 

80.500 

81 .676 

-1.176 

14 

82.000 

83.124 

-1.124 

15 

83.500 

84.473 

-0.973 

16 

85.000 

85.735 

-0.735 

17 

86.400 

86.920 

-0.520 

18 

87.600 

88.037 

-0.437 

19 

88.900 

89.094 

-0.194 

20 

90.100 

90.097 

0.003 

21 

91.200 

91 .051 

0.149 

22 

92.300 

91.960 

0.340 

23 

93.400 

92.829 

0.571 

24 

94.400 

93.661 

0.739 

25 

95.400 

94.459 

0.941 

26 

96.400 

95.226 

1.174 

27 

97.300 

95.964 

1.336 

28 

98.200 

96.675 

1.525 

29 

99.100 

97.361 

1.739 

30 

100.000 

98.023 

1.977 

TABLE  2.2 

TABLE  OF  RESIDUALS  FOR  LOG  PSEUDO-LEARNING  CURVE 
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. • The  data  in  Table  2.3  has  been  used  to  plot  the  cubic  pseudo-learning 

curve  of  Figure  2.2.  It  is  referred  to  as  a pseudo-learning  curve  because 
the  coordinates  are  now  "Calendar  Weeks"  as  the  independent  variable  and 
"Performance"  as  the  dependent  variable. 

The  model  for  the  cubic  pseudo-learning  curve  was  developed  using  the 
linear  regression  program  of  the  Statistical  Analysis  System  (SAS).  The 
model  has  the  following  form: 

P(t)  * 10.622  + 12.615  t - .59465  t2  + .0091986  t3 
where:  P(t)  = Performance,  percent 
t = Time,  calendar  weeks 

The  correlation  coefficient  for  this  model  is  r = .997.  Table  2.4  is  the 
table  of  residuals  of  the  model  compared  to  the  observed  values  (Table  2.3). 
Again,  it  can  be  noted  that  the  model  gives  an  extraordinarily  good  fit 
to  the  observed  data. 

The  cubic  pseudo-learning  curve  also  would  be  obtained  in  a work 
situation  where  the  operator  performs  only  one  task  and  is  not  interrupted 
in  his  performance  of  that  task.  An  example  of  this  type  of  continuous 
and  uninterrupted  job  position  would  be  a telephone  operator  who  does 
nothing  else  except  work  at  a switchboard.  The  next  section  of  this 
chapter  explains  a job  situation  in  which  the  operator  is  interrupted 
while  performing  his  primary  duties. 

2.1.3  Learn-Forget-Learn  (LFL)  Curve 

Little  research  has  been  done  in  the  area  of  Learn-Forget-Learn  curves. 


An  LFL  curve,  which  has  a shape  similar  to  the  curve  in  Figure  1.4,  occurs 
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Week 

Performance  [%) 

Week 

Performar 

1 

17.7 

16 

96.3 

2 

32.6 

17 

97.1 

3 

44.8 

18 

97.8 

4 

54.7 

19 

98.4 

5 

62.8 

20 

98.8 

6 

69.4 

21 

99.1 

7 

74.8 

22 

99.4 

8 

79.3 

23 

99.6 

9 

83.0 

24 

99.7 

10 

86.1 

25 

99.8 

11 

88.7 

26 

99.8 

12 

90.8 

27 

99.8 

13 

92.6 

28 

99.7 

14 

94.0 

29 

99.6 

15 

95.3 

30 

99.5 

TABLE 

2.3 

DATA  USED  TO  PLOT  CUBIC 

PSEUDO- LEARNING 

CURVE 
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Model:  P(t)  = 10.622  + 12.61 5t  - .59465t2  + .0091986t3 


Observed  Predicted 


Week 

Performance  Value 

Performance  Value 

Residual 

1 

17.700 

22.651 

-4.951 

2 

32.600 

33.546 

-0.946 

3 

44.800 

43.362 

1.438 

4 

54.700 

52.155 

2.545 

5 

62.800 

59.979 

2.821 

6 

69.400 

66.889 

2.511 

7 

74.800 

72.941 

1.859 

8 

79.300 

78.191 

1 .109 

9 

83.000 

82.692 

0.308 

10 

86.100 

86.501 

-0.401 

11 

88.700 

89.673 

-0.973 

12 

90.800 

92.263 

-1 .463 

13  • 

92.600 

94.325 

-1 .725 

14 

94.000 

95.916 

-1 .916 

15 

95.300 

97.090 

-1 .790 

16 

96.300 

97.902 

-1.602 

17 

97.100 

98.409 

-1 .309 

18 

97.800 

98.664 

-0.864 

19 

98.400 

98.724 

-0.324 

20 

98.800 

98.643 

0.157 

21 

99.100 

98.476 

0.624 

22 

99.400 

98.279 

1.121 

23 

99.600 

98.107 

1.493 

24 

99.700 

98.016 

1 .684 

25 

99.800 

98.059 

1.741 

26 

99.800 

98.293 

1.507 

27 

99.800 

98.773 

1.027 

28 

99.700 

99.553 

0.147 

29 

99.600 

100.690 

-1.090 

30 

99.500 

102.238 

-2.738 

TABLE  2.4 

TABLE  OF  RESIDUALS  FOR  CUBIC  PSEUDO-LEARNING  CURVE 
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when  an  operator  is  interrupted  while  working  at  his  primary  job  assignment 
and  is  assigned  to  another  task,  and  then,  after  a certain  period  of  time, 
he  returns  to  his  primary  duties.  The  curve  portrayed  in  Figure  1.4  depicts 
these  events  as  the  initial  learning  on  the  primary  task,  then  the  forgetting 
that  takes  place  during  the  interruption,  and  finally  the  resumed  learning 
of  the  task  after  the  operator  returns  to  the  job.  Instead  of  the  interruption 
taking  the  form  of  a change  in  job  assignments,  it  could  also  indicate  a 
period  of  absence  that  the  worker  is  away  from  his  primary  job,  i.e.  a week- 
end break  or  a vacation  for  the  worker.  The  amount  of  forgetting  that  takes 
place  during  a break  in  the  work  depends  on  how  much  the  worker  has  learned 
up  to  the  point  of  interruption  and  the  length  of  the  interruption  (13). 

Another  study  concerned  with  interrupted  learning  theorizes  that  a non- 
work interruption  (weekend  break  or  vacation)  is  not  the  same  as  a work 
interruption  (performing  another  task)  (9).  This  theory  still  has  to  be 
verified. 

The  LFL  curve  also  can  depicit  an  individual's  increasing  performance 
during  his  initial  training  for  a job  (first  section  of  curve),  his  de- 
creasing performance  caused  by  forgetting  since  the  initial  training 
(second  section  of  curve),  and  then  the  subsequent  increase  in  performance 
caused  by  retraining  for  the  job  (third  section  of  curve).  The  various 
work  phenomena  which  can  be  explained  by  LFL  curves  are  numerous  and  can 
be  easily  understood  by  using  this  type  of  curve.  A practical  example  of 
a situation  where  an  LFL  curve  could  be  applied  is  an  infantryman  who  re- 
ceived his  initial  training  on  the  use  and  firing  of  an  anti-tank  missile. 

After  the  initial  or  basic  training  period,  which  included  actual  firings 


31 


of  the  missile,  the  soldier  is  assigned  to  a unit  in  which  the  actual 
firing  of  the  missile  is  impossible.  In  all  probability  he  will  forget 
some  of  the  procedures  and  techniques  required  to  fire  the  missile  during 
his  assignment  to  this  particular  unit,  and  his  performance  in  regards  to 
missile  firings  will  decrease.  In  this  particular  instance,  performance 
can  be  construed  as  accuracy  in  hitting  the  target  i.e.  probability  of 
success.  He  is  then  retrained  on  the  missile  system  by  being  assigned  to 
a firing  range  where  he  can  perform  actual  firings  again,  and  his  performance 
level  should  increase  because  of  the  experience  he  received  on  the  range. 

See  Figure  2.3  for  a graphical  representation  of  the  infantryman's  training 
cycle  which  was  explained  above. 

The  example  of  interrupted  learning  presented  by  Carlson  and  Rowe  (6) 
has  an  operator  performing  a certain  task  for  a seven  week  period,  then 
being  assigned  to  perform  another  task  for  a period  of  12  weeks,  and  then 
returning  to  the  original  task  for  a period  of  11  weeks.  The  performance 
data  that  portrays  the  above  sequence  of  events  is  presented  in  Table  2.5 
and  is  plotted  in  Figure  2.4.  This  data  has  again  been  normalized  from  the 
original  data  presented  in  (6)  to  insure  that  the  performance  values  do  not 
exceed  100  percent.  Again,  this  normal ization  is  required  because  of  the 
definition  of  performance  which  was  explained  in  section  2.1. 

The  model  for  the  Learn-Forget-Learn  curve  was  developed  by  finding 
equations  for  each  of  the  three  sections  of  the  curve.  Each  equation  was 
formulated  using  analytical  methods.  The  model,  which  was  developed  using 
the  data  of  Table  2.5,  has  the  following  form: 





■■  
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(Probability  of  Success) 
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FIGURE  2.3 


EXAMPLE  LFL  CURVE  OF  TRAINING 
CYCLE  FOR  AN  INFANTRYMAN 
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P(t)  = 


*821.63  - — q-^-3  , Initial  Learning 


-15.690  + 13j-:98  , Forgetting 
t 


\ 108.47  - ^ , Resumed  Learning 


where:  P(t)  = Performance,  percent 
t = Time,  calendar  weeks 

Table  2.6  is  the  table  of  residuals  of  the  model  compared  to  the  observed 
values  (Table  2.5).  It  can  be  noted  in  Table  2.6  that  the  model  gives  a 
better  than  average  fit  to  the  observed  data,  and  the  pattern  of  residuals 
bears  this  out. 

As  was  mentioned  in  the  earlier  sections  of  the  chapter,  the  LFL  curve 
can  be  utilized  when  an  individual  operator  experiences  an  interruption  in 
his  work.  The  interruption  can  take  the  form  of  a work  interruption  (per- 
forming another  task)  or  a non-work  interruption  (weekend  break  or  vacation), 
and  it  should  be  stated  that  the  LFL  curve  is  applicable  to  both  types  of 
interruption. 

The  results  obtained  from  the  LFL  curve  and  the  two  pseudo-learning 
curves  (log  and  cubic)  of  this  chapter's  earlier  sections  will  be  utilized 
in  conjunction  with  a System  Effectiveness  model  in  Chapter  3.  The 
sensitivity  of  the  SE  model  to  the  three  different  curve  types  and 
changing  performance  values  of  each  curve  will  be  analyzed  and  discussed. 
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Week 

Performance  % 

Week 

Performance  (% 

1 

34.4 

16 

53.1 

2 

47.0 

17 

52.1 

3 

53.6 

18 

51.2 

4 

58.3 

19 

50.3 

5 

62.1 

20 

56.0 

6 

65.4 

21 

60.2 

7 

68.3 

22 

63.8 

8 

66.5 

23 

66.8 

9 

64.6 

24 

69.6 

10 

62.0 

25 

71  .9 

11 

60.0 

26 

74.0 

12 

58.2 

27 

76.0 

13 

56.7 

28 

77.9 

14 

55.4 

29 

79.6 

15 

54.1 

30 

81.2 

TABLE  2.5 

DATA  USED  TO  PLOT  LFL  CURVE 
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i 


Initial  Learning,  Model:  P ( t ) = 821.63  - 


787.23 

t'0225 


Observed  Predicted 

Week  Performance  Value  Performance  Value 


Residual 


1 

34.40 

34.40 

0 

2 

47.00 

46.58 

0.42 

3 

53.60 

53.62 

-0.02 

4 

58.30 

58.58 

-0.28 

5 

62.10 

62.40 

-0.30 

6 

65.40 

65.51 

-0.11 

7 

68.30 

68.12 

0.18 

Forqettinq,  Model: 

7 

P(t)  = 

68.30 

-15.690  + -3d8 
tmM 

68.30 

0 

8 

66.50 

65.60 

0.90 

9 

64.60 

63.40 

1.20 

10 

62.00 

61.40 

0.60 

11 

60.00 

59.70 

0.30 

12 

58.20 

58.10 

0.10 

13 

56.70 

56.70 

0 

14 

55.40 

55.40 

0 

15 

54.10 

54.30 

-0.20 

16 

53.10 

53.20 

-0.10 

17 

52.10 

52.20 

-0.10 

18 

51.20 

51 .30 

-0.10 

19 

50.40 

50.40 

0 

TABLE  2.6 


TABLE  OF  RESIDUALS  FOR  LFL  CURVE 
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Resumed  Learning,  Model:  P(t)  = 108.47  ^ — gg — 

Observed  Predicted 


Week 

Performance  Value 

Performance  Value 

Residual 

19 

50.40 

50.40 

0 

20 

56.00 

55.10 

0.90 

21 

60.20 

59.20 

1.00 

22 

63.80 

62.90 

0.90 

23 

66.80 

66.10 

0.70 

24 

69.60 

69.00 

0.60 

25 

71.90 

71.60 

0.30 

26 

74.00 

73.90 

0.10 

27 

76.00 

76.00 

0 

28 

77.90 

77.80 

0.10 

29 

79.60 

79.60 

0 

30 

81.20 

81.10 

0.10 

TABLE  2.6  continued 
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CHAPTER  3 

SYSTEM  EFFECTIVENESS  MODELS 

3.1  Introduction 

Before  proceeding  to  the  discussions  of  this  chapter,  the  term, 

"System  Effectiveness"  should  be  defined.  Gephart  and  Balachandran  (10) 
define  it  as:  "The  probability  that  the  man-machine  system  will  success- 
fully meet  an  operational  demand  and  fulfill  the  predetermined  mission  ob- 
jectives within  a given  mission  time  when  operated  under  stated  conditions." 
In  language  that  is  easier  to  understand,  System  Effectiveness  is  "the 
probability  that  a system  can  successfully  meet  an  operational  demand 
throughout  a given  time  period  when  operated  under  specified  conditions." 

In  most  cases,  System  Effectiveness  is  stated  in  probabilistic  form, 
i.e.  probability  of  system  success. 

This  chapter  is  organized  into  two  major  sections.  The  first  section 
consists  of  a summary  of  three  different  System  Effectiveness  models.  These 
three  models  will  be  identified  as  (1)  The  Modified  WSEIAC  Model,  (2)  The 
Navy  Model,  and  (3)  Lie's  Model.  After  the  summary,  a comparison  will  be 
made  between  the  three  models  with  differences  and  similarities  being  dis- 
cussed. 

The  sunmary  of  Lie's  model  will  be  in  more  detail  than  the  other  two 
models  because  Lie's  formulation  will  form  the  basis  for  the  next  major 
section  of  this  chapter.  Lie's  proposed  model,  is  very  similar  in  some 
aspects,  to  the  other  two  models,  but  it  addresses  two  areas  (environmental 
and  operator  impact  on  SE  ) which  were  not  mentioned  or  only  briefly 
explained  in  the  first  two  models.  The  area  of  operator  impact  on  System 
Effectiveness  is  of  major  interest  in  this  chapter. 


The  performance  values  obtained  from  the  two  pseudo-learning  curves 
(log  and  cubic)  and  the  Learn-Forget-Learn  curve  of  Chapter  2 will  be  used 
in  conjunction  with  Lie's  model  to  analyze  the  effect  that  the  various 
curve  forms  and  associated  levels  of  performance  have  on  the  value  of 
System  Effectiveness.  These  changes  in  the  values  of  Systan  Effectiveness 
then  will  be  analyzed  and  discussed  in  Chapter  4.  Also,  an  analysis  of  the 
behavior  of  the  LFL  curve  will  be  undertaken  in  the  last  section  of  this 
chapter. 

3.2  Comparison  of  System  Effectiveness  Models 

As  was  mentioned  in  the  preceding  section,  three  proposed  models  that 
attempt  to  quantify  System  Effectiveness  will  be  summarized  and  compared 
in  this  section  of  the  chapter.  The  three  models  will  be  referred  to  as 

(1)  The  Modified  WSEIAC  Model,  (2)  The  Navy  Model,  and  (3)  Lie's  Model. 

The  terms  used  in  the  discussion  and  analysis  are  defined  as: 

(1)  Availabil ity  - The  probability  that  the  system  is  in  an  "up" 
and  ready  state  at  the  beginning  of  the  mission  when  the  mission  occurs  at 
a random  point  in  time.  Availability  is  a function  of  the  reliability  and 
maintainability  characteristics  of  the  system. 

(2)  Reliability  - The  probability  that  an  item  will  perform  its  in- 
tended function  for  a specified  interval  under  stated  conditions. 

(3)  Maintainability  - The  probability  that  an  item  will  be  retained 
in  or  restored  to  a specified  condition  within  a given  period  of  time. 

(4)  Dependability  - The  probability  that,  given  the  system  was 
available,  it  will  continue  to  operate  throughout  the  mission  either  (1) 


without  a system-level  failure  (a  failure  that  causes  the  entire  system 
to  be  inoperable),  or  (2)  if  it  fails,  it  will  be  restored  to  operation  within 
some  critical  time  interval  which,  if  exceeded,  would  result  in  mission 
failure.  Dependability  is  also  a function  of  the  reliability  and  maintain- 
ability characteristics  of  the  system. 

(5)  Capability  - The  probability  that  the  system's  designed  performance 
will  allow  it  to  meet  mission  demands  successfully  assuming  that  the  system 
is  available  and  dependable. 

Now  that  some  of  the  more  important  terms  have  been  defined,  we  can 
proceed  to  the  summaries  of  the  three  models. 

3.2.1  The  Modified  WSEIAC  Model 

In  1963,  the  Weapon  System  Effectiveness  Industry  Advisory  Committee 
(WSEIAC)  was  formed  for  the  purpose  of  "providing  technical  guidance  and 
assistance  to  the  Commander,  Air  Force  Systems  Command,  in  the  development 
of  a technique  to  appraise  management  of  current  and  predicted  System 
Effectiveness  at  all  phases  of  system  life."  (10).  The  committee  theorized 
that  System  Effectiveness  was  a joint  probability  measure  expressed  as: 

SE  = (A)(0)(C)  (1) 

where:  SE  = System  Effectiveness 

A = Availability  of  the  system 
D = Dependability  of  the  system 
C s Capability  of  the  system 
and  where  A,  D,  and  C are  probability  statements. 
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They  further  stated  that  availability  (A)  may  be  obtained  as  a function 
of  the  state  readiness  of  the  system  and  the  utilization  of  the  system. 

This  meant  that  equation  (1)  was  then  transformed  to  be  the  following 
expression: 

SE  « (V)(W)(D)(C)  (2) 

where:  V = a measure  of  state  readiness  of  the  system. 

W = a measure  of  the  probability  of  utilizing  the 
system  given  the  state  of  the  system, 
and  where  V and  W are  probability  statements. 

In  1969,  Gephart  and  Balachandran  modified  equation  (2)  by  making 
the  following  changes: 

(1)  They  relabelled  V to  become  S (state  readiness),  and  relabelled 
W to  become  U (utility). 

(2)  They  relabeled  D as  RE-RE  (Reliability-Repairability). 

(3)  And  lastly,  they  proposed  that  the  capability  term  of  equation 
(2),  C,  could  be  expressed  as  the  product  of  "adequacy  of  personnel".  A, 
and  "capability  of  hardware",  CH. 

With  the  above  changes  being  made  to  equation  (2),  it  would  then  take 
the  following  form: 

SE  = ( S ) (U ) (RE-RE ) ( A) (CH ) (3) 

where  S,  U,  RE-RE,  A,  and  CH  are  probability  statements. 

The  major  modification  that  Gephart  and  Balachandran  made  to  the 
original  WSEIAC  model,  equation  (2),  was  to  partition  the  capability  of 
the  system,  C,  intu:  (a)  that  which  was  contributed  by  the  hardware  of 
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the  system,  and  (b)  that  which  was  attributable  to  the  human  factor  (oper- 
ator). By  partitioning  the  capability  of  the  system,  they  realized  that 
the  performance  of  the  operator  has  a definite  impact  on  System  Effective- 
ness. In  earlier  studies,  operator  performance  was  assumed  to  have  a con- 
stant value  of  100  per  cent  i.e.,  the  operator  was  totally  reliable.  In 
actuality,  an  operator's  performance  is  very  seldom  totally  reliable  and 
therefore,  this  assumption  led  to  miscalculations  of  System  Effectiveness. 

They  defined  "adequacy  of  personnel"  as  the  conditional  probability 
that  the  personnel  will  perform  at  their  level  of  proficiency,  given  that 
the  hardware  component  of  the  system  is  in  a given  state.  They  assumed 
in  the  model  that  the  variable  which  describes  operator  performance  follows 
a normal  distribution.  They  further  stated  that  the  parameters  of  this  dis- 
tribution can  be  obtained  from  the  training  programs  or  proficiency  evalu- 
ations of  a sample  from  the  relevant  population  of  subjects.  These  para- 
meters then  can  be  used  in  the  Systems  Effectiveness  simulation. 

However,  they  did  not  detail  how  to  obtain  useable  human  performance 
data  from  the  training  programs  or  proficiency  evaluations.  In  oti.e*-  words, 
they  presented  no  analytical  method  which  could  be  used  to  extract  data 
from  the  training  programs/proficiency  evaluations,  n/ithout  being  able  to 
extract  human  performance  data  from  the  sources  they  mentioned,  the  human 
performance  portion  of  their  Capability  term  is  useless.  Because  of  this, 
no  operator  performance  data  was  utilized  in  the  example  problem  they  pre- 
sented in  their  paper.  Therefore,  in  essence,  Gephart  and  Bal achandran 
outlined  the  requirement  for  including  an  operator  pe-fonnance  term  in  the 


calculation  of  System  Effectiveness,  but  did  not  explain  how  to  obtain 
operator  performance  data  which  could  then  be  utilized  in  the  System 
Effectiveness  simulation. 


3.2.2  The  Navy  Model 

The  Navy  Model  for  System  Effectiveness  was  obtained  from  a proposed 
revision  of  the  "Navy  System  Effectiveness  Manual"  which  was  written  by 
D.  T.  Hanifan  (14).  This  model  is  very  similar  to  the  Modified  WSEIAC 
Model  presented  in  the  last  section.  In  the  manual,  Hanifan  states  that 
"the  effectiveness  of  a system  depends  on  its  availability,  dependability, 
and  capaoi 1 ity  in  relation  to  the  mission."  This  statement  expresses  the 
same  formulation  for  System  Effectiveness  as  was  presented  by  equation  (1) 
of  the  last  section,  i . e . : 

SE  - (A)(0)(C)  (1) 

Hanifan  goes  on  to  state  that  the  three  terms  of  the  model  are  mutually 
exclusive,  and  great  care  should  be  exercised  in  modeling  to  guarantee  that 
the  same  data  are  not  included  in  more  than  one  term  of  the  model. 

As  was  the  case  for  the  Modified  WSEIAC  Model,  Hanifan  says  that  the 
"Capability"  term,  C,  of  equation  (1)  can  be  partitioned  into  a term 
which  is  contributed  by  the  hardware  of  the  system  and  a term  which  is 
attributable  to  the  performance  of  the  operator.  Usually,  Capability  is 
less  than  theoretical  computations  or  test  results  because  the  human  per- 
formance part  of  the  Capability  term  may  have  been  overestimated  or  even 
assumed  to  be  1.0  (which  is  the  assumption  when  the  human  performance  term 
is  effectively  left  out).  Hanifan  says  that  because  of  the  above  assumption, 
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"the  effectiveness  modeler  must  usually  modify  the  system  performance  numbers 
obtained  from  hardware  designers  in  order  to  obtain  a more  accurate  estimate 
of  total  system  Capability."  Because  of  the  difficulty  in  obtaining  suitable 
human  performance  data,  estimates  for  this  data  must  frequently  be  substi- 
tuted for  empirically-obtained  data.  He  says  some  of  the  human  performance 
parameters  can  be  estimated  from  experimental  data  or  operational  records, 
but  many  are  at  present  known  only  qualitatively  and  their  effects  must  be 
estimated  on  the  basis  of  judgement.  Too  often  the  tendency  is  to  leave 
the  human  performance  data  completely  out  of  the  model. 

As  in  the  last  section,  the  author  notes  the  importance  of  including 
an  operator  performance  term  in  the  formulation  of  System  Effectiveness, 
but  gives  no  concrete  method  for  obtaining  data  which  can  be  used  in  the 
operator  performance  term.  In  addition  to  not  presenting  any  concrete 
metnod  for  obtaining  this  data,  he  does  not  present  a method  for  estimating 
the  data  that  would  be  required  to  formulate  the  human  performance  term. 

He  discusses  the  importance  of  human  performance,  but  that's  all. 

3.2.3  Lie's  Model 

The  formulation  of  this  model  is  contained  in  Lie's  doctorial  disser- 
tation (16).  Lie  developed  numerous  models  that  attempted  to  quantify 
Mission  Effectiveness  (ME  ).  To  be  consistent  with  the  terminology  used 
in  the  preceding  two  models,  we  note  that  System  Effectiveness  is  Mission 
Effectiveness,  hence  Lie's  term  of  Mission  Effectiveness  will  be  labelled 
System  Effectiveness. 
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Lie’s  general  formulation  of  System  Effectiveness  is: 


(4) 


where:  (SE).. 

1 J 


j 

and  where  all  th 


= System  Effectiveness  of  unit  i for  mission  j 

= Availability  of  the  hardware  component  of  unit  i 
at  the  start  of  mission  j 

= Availability  of  the  operator  component  of  unit  i 
at  the  start  of  mission  j 

= Mission  reliability  of  the  hardware  component  of 
unit  i for  mission  j 

= Mission  reliability  of  the  operator  of  unit  i for 
mission  j 

= Performance  of  unit  i during  mission  j for  a given 
status  of  the  environment 

= Performance  of  the  operator  of  unit  i during 
mission  j 

e terms  of  equation  (4)  are  probability  statements. 


Equation  (4)  is  comparable  to  equation  (1)  of  the  preceding  sections  except 
for  one  major  deviation  - the  "Capability"  portion  of  the  model  represented 
by  equation  (1)  is  now  expressed  by  a term  for  environmental  impact  on  SE 


and  a term  that  deals  with  operator  performance  and  its  effect  on  SE. 

Lie  states  that  "the  performance  of  a unit  is  dependent  upon  the 
status  of  the  environment",  i.e.  a better  performance  of  the  unit  is 
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expected  in  a good  weather  condition  than  in  a bad  weather  condition 
(cold  winter,  stormy  night,  etc.).  Lie  classified  the  status  of  the  en- 
vironment as  excellent,  good,  fair,  poor,  etc.  He  also  expressed  the  idea 
that  the  performance  of  a unit  is  dependent  upon  the  performance  of  the 
operator,  and  performance  is  a function  of  the  quality  of  the  operator  and 
the  retraining  period.  He  assumed  that  the  performance  of  the  operator  of 
unit  i can  be  expressed  by  the  following  functional  form: 

y,  = (y^  + (y2)i  e 1 (5) 

where:  y^  = Probability  of  the  mission  success  as  a function  of 
the  operator  effect  of  unit  i at  time  t 
(y-])-j  = Steady-state  probability  of  the  mission  success  as  a 
function  of  the  operator  effect  of  unit  i 
(y -j ) -j+(y2 ^ i = Iiritial  peak  probability  of  the  mission  success  as  a 
function  of  the  operator  effect  of  unit  i 
8,j  = Decreasing  rate  of  the  probability  of  the  mission 

success  as  a function  of  the  operator  effect  of  unit  i 
t = time,  hours 

If  the  retraining  of  the  operator  is  performed  every  time  units,  and  if 
every  retraining  brings  the  performance  up  to  the  initial  level,  then  the 
performance  of  the  operator  of  unit  i may  be  represented  as  shown  in 
Figure  3.1 . 


Lie  states  that  equation  (5)  is  one  of  a variety  of  functional  forms 
for  the  operator  performance  that  can  be  assumed.  Equation  (5)  is  an 
expression  for  operator  performance  and  could  be  termed  an  "LFL"  curve. 


Performance 

(Probability  of  Success) 


r-  — 1 
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Lie  assigned  arbitrary  values  to  the  parameters  of  equation  (5)  when  tne 
expression  was  utilized  in  his  simulation  of  the  model.  He  provided  no 
explanation  as  to  why  he  used  the  particular  equation  he  did,  equation  (5), 
or  why  he  chose  the  particular  values  for  the  parameters  that  he  did.  He 
did  not  mention  how  values  for  the  parameters  could  be  obtained  from 
training  programs,  proficiency  evaluations,  etc. 

Using  the  equations  for  operator  performance  developed  in  Chapter  2 
from  the  learning  and  LFL  curves  in  place  of  equation  (5),  the  simulation 
program  for  System  Effectiveness  (the  simulation  program  for  SE  is  listed 
in  Appendix  A)which  Lie  developed  was  run  to  determine  what  effect  the 
various  curve  forms  would  have  on  the  overall  System  Effectiveness.  The 
results  of  the  various  simulation  runs  will  be  outlined  in  section  3.3.  with 
subsequent  conclusions  made  in  Chapter  4. 

3.2.4  Comparison  of  Models 

It  should  be  evident  from  the  three  preceding  sections  that  the  three 
models  are  very  similar  in  most  aspects  with  the  only  differences  being  in 
the  interpretation  of  the  "Capability"  term  of  equation  (1).  The  Modified 
WSEIAC  Model  and  the  Havy  Model  are  almost  identical  in  their  formulation  of 
System  Effectiveness  while  the  model  of  Lie's  differs  in  the  make-up  of 
the  "Capability"  term.  Lie  also  developed  his  model  for  Mission  Effective- 
ness while  the  other  two  were  formulated  in  terms  of  System  Effectiveness, 
but  both  terms  ( SE  and  ME  ) employ  the  same  concepts. 

The  one  fact  that  should  be  brought  out  in  this  comparison  is  that  none 
of  the  models  provided  any  definitive  data  on  human  performance.  Also,  no 
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guidance  at  all  was  provided  in  regards  to  obtaining  analytical  expressions 
that  could  be  used  to  quantify  human  performance.  Lie  did  the  most  work 
in  this  area,  but  he  was  deficient  in  the  explanation  of  the  equation  he 
used,  and  he  did  not  list  possible  sources  of  human  performance  data.  As 
far  as  the  operator  performance  term  in  each  of  the  models,  there  was  much 
discussion  about  what  should  be  done  in  this  area,  but  no  one  gave  any 
direction  that  could  be  followed  when  trying  to  quantify  human  performance 
levels.  This  paper  will  give  insight  into  the  collecting  of  human  per- 
formance data, development  of  analytical  expressions  for  the  performance, 
and  their  subsequent  use  in  effectiveness  simulation  models. 

3.3  Lie's  Simulation  Model  with  Modified  Operator  Performance  Term 


Lie  (16)  developed  a number  of  simulation  models  that  attempted  to 
quantify  System  Effectiveness.  The  models  varied  according  to  the  constraints 
and  assumptions  that  were  applied  to  the  various  systems.  The  particular 
simulation  model  that  will  be  utilized  in  this  section  was  developed  for 
a system  which  was  required  to  carry  out  various  types  of  missions.  In 
this  particular  model,  each  mission  type  is  characterized  by  the  maximum 
allowable  time  that  determines  the  success  of  a given  mission  type.  Lie 
described  the  logic  of  this  model  in  the  following  way: 


"For  a given  type  of  a mission  to  be  successful,  the  system  is  re- 
quired to  be  available  at  the  start  of  a mission,  and  the  system 
must  complete  its  mission  within  the  maximum  allowable  duration 
of  time  that  this  given  mission  type  specifies  without  any  failure 
during  this  period.  If  the  system  cannot  accomplish  a mission 
within  the  specified  duration  of  time,  the  mission  is  terminated 
at  this  point  and  is  considered  to  be  failed  even  though  the  system 
is  still  operable.  Failures  of  the  system  are  induced  by  both  the 
hardware  itself  and  the  operator.  Furthermore,  the  effects  of  the 
environment  and  the  operator  are  reflected  in  the  mission  duration. 
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In  other  words,  poor  environmental  conditions  and  poor  operator 
performance  are  assumed  to  make  the  actual  mission  duration 
longer  than  the  mission  duration  under  ideal  conditions.  Thereby, 
adverse  effects  of  the  environment  and  the  operator  tend  to  re- 
duce the  probability  of  mission  success,  i.e.  System  Effectiveness." 

Hopefully,  this  short  synopsis  of  the  system  will  help  explain  the  simu- 
lation model  for  this  particular  system.  Again,  a printout  of  the  simu- 
lation program  used  in  this  section  is  listed  in  Appendix  A. 

The  section  of  the  simulation  program  which  was  of  major  interest 
in  this  paper  dealt  with  the  operator  performance  term,  OP ( I , J ) , and  its 
formulation.  In  the  simulation  program  (Appendix  A),  cards  number  177 
through  206  calculated  the  operator  performance  for  unit  i and  mission  j, 
0P(I,J),  and  printed  the  various  values  of  0P(I,J)  in  the  output. 

Equation  (5)  of  section  3.2.3  was  used  by  Lie  to  express  the  operator  per- 
formance of  unit  i and  mission  j in  the  simulation  program.  He  assumed 
the  following  values  for  the  parameters  of  equation  (5): 

y1  = .8,y2  = .2,  6 = .0014,  T = 2160  hours 

where  y-|,  and  s are  probability  values. 

When  the  above  values  were  used  in  equation  (5),  and  a total  of  50  missions 
were  simulated  for  a single  unit,  the  operator  performance  for  each  of  the 
50  missions  was  calculated  to  be  the  values  in  Table  3.3.  Using  the  values 
of  Table  3.3,  the  overall  System  Effectiveness  for  the  unit  turned  out  to 
be  .52  after  all  the  calculations  of  the  simulation  were  completed. 

The  equations  for  operator  performance,  which  were  developed  in 
Chapter  2 from  the  three  different  curves,  were  substituted  into  Lie's 
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simulation  program  in  place  of  his  expression  for  0P(I,J)  to  determine 
what  values  of  operator  performance,  and  subsequently,  what  value  of 
System  Effectiveness  would  result.  The  equations  from  Chapter  2 are  as 
follows : 

Log  Pseudo-Learning  Curve 
0P(I,J)  = 31 .534  + 19.549  log  t 

Cubic  Pseudo-Learning  Curve 

OP(I.J)  = 10.622  + 12.615  t - 0.59465  t2  + 0.0091986  t3 


Learn-Forget-Learn  Curve 


Initial  Learning,  0P(I,J)  = 821.63  - 


787.23 
* .02 


Forgetting,  OP(I.J)  = -15.690  + L4j- 

Resumed  Learning,  0P(I,J)  = 108.47  - 

Note  that  "t"  in  the  above  equations  was  replaced  by  "CTMS(I,J)"  when  the 
equations  were  utilized  in  the  simulation  program.  "CTMS(I,«3)"  stands 
for  the  mission  start  time  expressed  in  clock  time  for  unit  i and  mission  j. 
This  transformation  was  made  because  the  simulation  program  operates  on 
a clock  time  basis,  and  "CTMS(I,J)"  is  Lie's  clock  time  term  which  is 
equivalent  to  "t". 

Each  of  the  three  equations  were  substituted  into  Lie's  program  with 
each  equation  being  run  separately  in  the  simulation.  The  resulting  values 
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of  operator  performance  for  the  50  missions  obtained  from  the  Log  Pseudo- 
Learning  curve,  Cubic  Pseudo-Learning  curve,  and  Learn-Forget-Learn 
curve  are  listed  in  Tables  3.4,  3.5,  and  3.6  respectively.  The  subsequent 
values  for  overall  System  Effectiveness  after  the  three  equations  from 
Chapter  2 and  Lie's  equation  were  utilized  are  tabulated  in  Table  3.7. 

The  values  of  0P(I,J)  = .50  and  1.0  were  also  used  in  the  simulation  to 
obtain  a range  for  the  System  Effectiveness  values.  The  resulting  SE 
values  for  these  two  constant  operator  performance  terms  are  also  listed 
in  Table  3.7. 

3.4  Analysis  of  Learn-Forget-Learn  (LFL)  Curve 

The  mean  performance  value  for  the  Learn-Forget-Learn  curve.  Figure  2.4, 
is  calculated  in  this  section  along  with  the  corresponding  mean  value  of 
System  Effectiveness  which  results  when  the  performance  mean  is  utilized 
in  Lie's  simulation  program.  Also,  in  this  section,  the  average  operator 
performance  is  calculated  for  sample  missions  taken  from  the  total  of  50 
missions.  The  values  of  the  above  mean  performance  figures  and  their  re- 
sulting System  Effectiveness  indexes  will  be  compared  and  discussed  in 
Chapter  4. 

3.4.1  Mean  Performance  of  LFL  Curve 

The  mean  performance  value  for  the  Learn-Forget-Learn  curve  is  cal- 
culated by  integrating  the  three  separate  segments  of  the  curve  over  the 
time  periods  that  they  cover  and  then  dividing  the  sum  of  the  integration 
results  by  the  total  time  period  for  which  the  curve  is  effective.  The  LFL 
curve  is  portrayed  in  Figure  2.4  with  the  equation  for  each  of  the  three 
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sections  of  the  curve  listed  above  the  portion  of  the  curve  to  which  it 
applies.  The  integration  which  yields  the  area  below  the  LFL  curve  is  as 
follows: 


-L-)dt  + / 9(-l  5,690  + 13p/98)dt  + /3°(1 08.47 
t,u*  7 19 


7480 

X65 


)dt 


From  the  above  integration,  the  total  area  below  the  curve  is  found  to  be: 


339.18  + 690.41  + 757.13  = 1786.72  weeks 


When  this  total  area  is  divided  by  the  total  time  interval  for  which  the 
curve  is  effective,  the  mean  operator  performance,  OTP.,  will  result: 

OTpT  = -7-8q--  = 61 .6% 


By  using  the  predicted  performance  values,  Pi , from  the  table  of 

residuals  for  the  LFL  curve,  Table  2.6,  in  conjunction  with  the  value  of 

P,  the  standard  deviation  that  pertains  to  the  mean  performance,  s , can 

O.P. 

be  calculated  as  follows: 


s_ 

O.P. 


30 

[ (P,-P)‘ 

i=l  1 


n-1 


1/2 


f 31 97 .991 1/2 
l 30-1  j 


10.50% 


The  values  of  O.P.  and  s will  be  utilized  in  Chapter  4 for  comparino 

O.P. 

various  values  obtained  from  the  LFL  curve.  Before  proceeding  to  the  next 
section,  it  should  be  noted  that  the  value  of  P obtained  above  is  only  valid 
for  a large  number  of  missions,  and  it  should  not  be  utilized  when  estimating 
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the  performance  for  a single  mission  or  a small  sample  of  missions. 

The  reasoning  behind  this  statement  will  be  demonstrated  in  Chapter  4. 

When  the  value  of  O.P.  is  substituted  into  Lie's  simulation  program, 
the  System  Effectiveness  turns  out  to  be  .38. 

3.4.2  Mission  Sampling 

To  study  the  sensitivity  of  the  operator  performance  at  a point  in 
time  versus  the  overall  performance  value,  four  samples  (r.=4),  each  of 
size  five,  of  missions  out  of  the  total  of  50  simulated  rrrssion  were  taken. 
The  four  samples  were  taken  from  around  the  10^,  20th,  2.  , and  40^ 

mission  intervals.  As  was  mentioned  earlier,  each  sample  will  consist 
of  five  observations.  For  example,  the  sample  for  the  10th  mission  interval 
is  comprised  of  five  observations,  i.e.  readings  from  mission  numbers  8,  9, 
10,  11,  and  12.  From  the  computer  output  of  Lie's  simulation  which  was 
run  using  the  equations  for  the  LFL  curve,  operator  performance  values 
(O.P.)  for  each  individual  mission  are  obtained.  The  average  operator  per- 
formance for  each  of  the  four  samples  is  then  calculated  as  depicted  in 
Table  3.1. 

When  each  of  the  0 . P . . (i  = 10,20,30,40)  are  averaged  together,  the 
resulting  value,  the  grand  mean  (O.P. ) of  the  four  samples,  turns  out  to  be: 

0T7  = 60* 

Note  that  the  value  of  60%  obtained  in  this  section  is  very  close  to  the 
value  of  62%  obtained  in  section  3.4.1.  This  result  is  only  logical  because 
the  two  mean  values  were  obtained  from  the  same  population.  The  small 
difference  in  their  values  is  the  result  of  the  two  different  methods 


utilized  to  calculate  the  means. 
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By  using  the  values  of  O.P.^  ( i =1 0,20,30, 40 ) in  conjunction  with  the 
value  of  O.P.,  the  standard  deviation  that  pertains  to  the  mean  performance 
calculated  in  this  section  is  found  as  follows: 


The  values  of  O.P.  and  s which  were  calculated  in  this  section  will  be 
utilized  in  Chapter  4 for  comparing  various  values  obtained  from  the  LFL 
curve. 

For  each  of  the  four  mission  intervals  sampled,  the  average  System 
Effectiveness  ( SE  ) for  each  interval  can  be  calculated  by  using  the 
computer  outp"t  of  Lie's  simulation.  It  should  be  noted  that  System 
Effectiveness  can  only  have  values  of  0 or  1 , i.e.  the  mission  either 
fails  or  it  is  successful.  The  calculations  for  average  System  Effectiveness 
are  depicted  in  Table  3.2. 

When  each  of  the  "SE^  (i=10,20,30,40)  are  averaged  together,  the 
resulting  value,  "SE-  , turns  out  to  be: 

5T  = .40 

Note  that  the  average  value  for  System  Effectiveness  obtained  in  this 
section,  .40,  and  for  the  entire  period  found  in  the  preceding  section, 

.38,  are  very  close,  as  well  they  should  be,  because  they  were  obtained 
from  the  same  population  of  values.  The  small  difference  in  their  values 
results  from  the  two  different  methods  utilized  to  calculate  the  averages. 
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One  final  item  should  be  mentioned  in  this  section.  Since  the  results 
of  this  section  were  calculated  by  using  the  output  from  Lie's  simulation 
and  since  the  form/content  of  the  output  was  not  listed  here.  Lie's  dis- 
sertation (16)  can  be  consulted  for  further  explanation. 
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10th  Mission  Interval  20th  Mission  Interval 


Mission  No. 

O.P. 

Mission  No. 

O.P. 

8 

61 

18 

55 

9 

61 

19 

54 

10 

59 

20 

54 

11 

58 

21 

53 

12 

57 

22 

52 

o.p.io= 

:59 

0.P.2q=54 

30th 

Mission 

Interval 

40th 

Mission  Interval 

Mission  No. 

O.P. 

Mission  No. 

O.P. 

28 

50 

38 

67 

29 

52 

39 

69 

30 

56 

40 

70 

31 

58 

41 

71 

32 

60 

42 

72 

O.P.40=70 


TABLE  3.1 

CALCULATIONS  OF  OTpT. 
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10  Mission  Interval 


20^  Mission  Interval 


Mission  No. 


Mission  No. 


SEiq-.20 


SE20=.60 


j.  L 

30l  Mission  Interval 


i.  L. 

40Ln  Mission  Interval 


Mission  No. 


Mission  No. 


se30.o 


TABLE  3.2 


CALCULATIONS  OF  SE, 


*40= -80 
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L 


ssion  No. 

Operator  Performance 

Mission  No. 

Operator  Performance 

I 

.99 

26 

.95 

2 

.99 

27 

.95 

3 

.98 

28 

.95 

4 

.98 

29 

.94 

5 

.97 

30 

.94 

6 

.97 

31 

.94 

7 

.97 

32 

.94 

8 

.97 

33 

.94 

9 

.97 

34 

.94 

10 

.97 

35 

.94 

11 

.96 

36 

.94 

12 

.96 

37 

.94 

13 

.96 

38 

.94 

14 

.96 

39 

.93 

15 

.96 

40 

.93 

16 

.96 

41 

.93 

17 

.96 

42 

.93 

18 

.96 

43 

.93 

19 

.96 

44 

.93 

20 

.95 

45 

.93 

21 

.95 

46 

.93 

22 

.95 

47 

.93 

23 

.95 

48 

.93 

24 

.95 

49 

.93 

25 

.95 

50 

.92 

TABLE  3. 

3 

OPERATOR  PERFORMANCE 

VALUES  RESULTING 

FROM  LIE'S  EQUATION 


r 
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Mission  No. 

Operator  Performance 

Mission  No. 

Operator  Performance 

I 

.45 

26 

.88 

2 

.59 

27 

.89 

3 

.67 

28 

.89 

4 

.72 

29 

.89 

5 

.74 

30 

.90 

6 

.74 

31 

.91 

7 

.76 

32 

.91 

8 

.77 

33 

.91 

9 

.77 

34 

.92 

10 

.79 

35 

.92 

11 

.80 

36 

.92 

12 

.81 

37 

.93 

13 

.82 

38 

.93 

14 

.82 

39 

.94 

15 

.82 

40 

.94 

16 

.83 

41 

.94 

17 

.83 

42 

.95 

18 

.83 

43 

.95 

19 

.84 

44 

.95 

20 

.85 

45 

.95 

21 

.86 

46 

.96 

22 

.87 

47 

.96 

23 

.87 

48 

.96 

24 

.87 

49 

.96 

25 

.88 

50 

iO 

oo 

TABLE  3.4 


GPERATOR  PERFORMANCE  VALUES  RESULTING 
FROM  LOG  PSEUDO-LEARNING  EQUATION 
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Mission  No. 

Operator  Performance 

Mission  No. 

Operator  Performance 

1 

.34 

26 

.99 

2 

.52 

27 

.99 

3 

.67 

28 

.99 

4 

.78 

29 

.99 

5 

.81 

30 

.99 

6 

.82 

31 

.99 

7 

.85 

32 

.98 

8 

.87 

33 

.98 

9 

.88 

34 

.98 

10 

.91 

35 

.98 

11 

.92 

36 

.98 

12 

.94 

37 

.98 

13 

.94 

38 

.98 

14 

.95 

39 

.98 

15 

.95 

40 

.98 

16 

.95 

41 

.98 

17 

.96 

42 

.98 

18 

.96 

43 

.98 

19 

.97 

44 

.98 

20 

.98 

45 

.98 

21 

.98 

46 

.99 

22 

.98 

47 

.99 

23 

.98 

48 

.99 

24 

.99 

49 

.99 

25 

.99 

50 

1.00 

TABLE  3.5 


OPERATOR  PERFORMANCE  VALUES  RESULTING 
FROM  CUBIC  PSEUDO-LEARNING  EQUATION 
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ssion  No. 

Operator  Performance 

Mission  No. 

Operator  Pp 

1 

.47 

26 

.51 

2 

.59 

27 

.51 

3 

.66 

28 

.50 

4 

.66 

29 

.52 

5 

.64 

30 

.56 

6 

.64 

31 

.58 

7 

.62 

32 

.60 

8 

.61 

33 

.61 

9 

.61 

34 

.62 

10 

.59 

35 

.63 

11 

.58 

36 

.65 

12 

.57 

37 

.66 

13 

.57 

38 

.67 

14 

.56 

39 

.69 

15 

.56 

40 

.70 

16 

.56 

41 

.71 

17 

.55 

42 

.72 

18 

.55 

43 

.73 

19 

.54 

44 

.73 

20 

.54 

45 

.74 

21 

.53 

46 

.75 

22 

.52 

47 

.75 

23 

.52 

48 

.77 

24 

.52 

49 

.77 

25 

.52 

50 

.81 

TABLE  3.6 

OPERATOR  PERFORMANCE  VALUES  RESULTING 
FROM  LEARN-FORGET-LEARN  EQUATION 
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Operator  Performance  Equation 

System  Effectiveness 

Lie's 

.52 

Log  Pseudo-Learning 

.56 

Cubic  Pseudo-Learing 

.58 

Leam-Forget-Learn  (LFL) 

.42 

Constant  value  of  .50 

.32 

Constant  value  of  1 .0 

.70 

TABLE  3.7 

SYSTEM  EFFECTIVENESS  VALUES  FOR  VARIOUS 
OPERATOR  PERFORMANCE  EQUATIONS 
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CHAPTER  4 

RESULTS  AND  CONCLUSIONS 

4.1  Introduction 

In  this  chapter,  the  conclusions  of  this  work  will  be  discussed.  Also 
contained  in  this  chapter  is  a discussion  of  the  sensitivity  of  Lie's 
System  Effectiveness  model  when  the  various  operator  performance  equations 
are  utilized  in  the  simulation.  There  is  also  an  analysis  comparing  the 
mean  operator  performance  for  a small  number  of  missions  in  the  same  region 
of  the  LFL  curve  with  the  mean  performance  value  for  the  entire  cycle  of  the 
LFL  curve.  The  last  section  of  this  chapter  will  outline  the  requirement 
for  possible  future  investigations  in  the  area  of  training/learning  and 
their  impact  on  operator  performance  and  the  sub'  'ent  effect  of  operator 
performance  on  the  effectiveness  of  a system. 

4.2  Summary  and  Discussion  of  Results 

In  this  section,  the  findings  of  Chapter  3 are  summarized  and  analyzed. 
The  sections  of  Chapter  3 which  are  of  interest  here  are:  3.3,  3.4.1,  and 
3.4.2. 

4.2.1  Sensitivity  Analysis  of  Lie's  SE  Model 

Referring  back  to  Table  3.7  which  lists  values  of  System  Effectiveness 
for  the  various  expressions  of  operator  performance,  it  can  be  seen  that 
the  values  for  System  Effectiveness  definitely  depend  upon  which  equation 
for  operator  performance  is  utilized  in  the  simulation  program.  In  other 
words.  System  Effectiveness  is  a function  of  operator  performance  when  the 
expression  for  operator  performance  is  used  in  conjunction  with  the  simu- 
lation. 
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The  resultant  System  Effectiveness  values  for  the  Log  Pseudo- 
Learning  and  Cubic  Pseudo-Learning  curves  (.56  vs  .58)  are  relatively  close 
together  as  would  be  expected  by  comparing  the  shapes  of  the  two  curves  in 
Figures  2.1  and  2.2.  They  both  represent  increasing  performance  functions 
with  the  only  difference  being  that  the  Cubic  curve  reaches  the  asymptote 
of  1.0  faster  than  the  Log  curve.  This  explains  why  the  System  Effectiveness 
value  of  the  Cubic  curve  (.58)  is  slightly  larger  than  that  of  the  Log 
curve  (.56). 

The  System  Effectiveness  index  corresponding  to  the  Learn-Forget-Learn 
(LFL)  curve  (SE  = .42)  is  significantly  less  than  the  SE  values  of  the  Log 
and  Cubic  curves  (.56  and  .58).  The  reasoning  behind  this  difference  in 
values  can  again  be  explained  by  comparing  the  shapes  of  the  three  curves 
(Figures  2.1,  2.2,  and  2.4).  The  LFL  curve  portrays  an  increesing-decreasing- 
increasing  function  of  performance  while  the  other  two  curves  are  strictly 
increasing  functions  of  performance.  Because  the  LFL  curve  has  a decreasing 
performance  section,  this  explains  the  smaller  value  of  System  Effectiveness 
for  this  particular  curve. 

Lie  s expression  for  operator  performance,  y = y^  + y^e  , that  he 
utilized  in  the  simulation  yielded  a System  Effectiveness  value  of  .52. 

Even  though  this  expression  is  a decreasing  function  of  performance  between 
retraining  periods,  it  still  produces  a relatively  high  index  of  System 
Effectiveness.  This  is  because  the  curve  starts  at  a performance  value 
close  to  100  per  cent  and  decreases  from  there  to  a operator  performance 
value  of  92  per  cent  which  is  large  compared  to  the  performance  values 
of  the  Log  and  Cubic  curves. 
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When  the  constant  values  of  .50  and  1.0  for  operator  performance  are 
utilized  in  the  simulation,  they  produced  a range  of  System  Effectiveness 
values  from  .32  to  .70.  It  should  be  noted  that  the  other  four  values  for 
System  Effectiveness  fall  between  the  values  of  .32  and  .70. 

From  the  above  discussion,  it  is  clear  that  System  Effectiveness  is 
very  sensitive  to  the  various  equations  that  express  operator  performance. 

4.2.2  Comparison  of  Mean  Performance  Values  Obtained  from  LFL  Curve 
The  mean  operator  performance  values  calculated  in  sections  3.4.1 
and  3.4.2  are  summarized  in  Table  4.1.  Note  that  the  grand  mean  calculated 
from  the  means  of  the  four  samples  is  approximately  the  same  as  the  mean 
calculated  by  the  integration  method.  As  was  mentioned  in  section  3.4.2, 
this  result  is  not  surprising  because  the  sample  population  of  values  were 
utilized  to  calculate  the  two  means.  The  same  logic  applies  to  the  fact 
that  the  two  System  Effectiveness  values  are  approximately  the  same. 

The  most  important  result  obtained  from  the  method  of  mission  interval 
samples  is  that  the  means  of  the  individual  samples  are,  in  most  cases, 
significantly  different  than  the  overall  mean  value;  that  is,  .55,  .54, 

.57,  and  .70  are  significantly  different  than  .60.  This  implies  that  the 
overall  mean  can  be  used  to  estimate  the  average  System  Effectiveness 
if  a large  number  of  missions  are  to  be  considered,  but  if  the  average 
System  Effectiveness  for  a small  interval  of  missions  is  required,  the 
overall  mean  performance  value  can  not  be  utilized.  When  a small  interval 
of  missions  is  to  be  studied,  the  average  operator  performance  has  to  be 
obtained  by  consulting  the  portion  of  the  LFL  curve  which  applies  to  the 
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mission  interval  under  study.  The  average  operator  performance  value 
obtained  from  the  applicable  portion  of  the  LFL  curve  then  can  be  utilized 
to  calculate  the  average  System  Effectiveness  for  the  specific  interval  of 
missions  under  consideration. 

4.2.3  Summary  of  Results 

Table  3.7  demonstrated  that  the  System  Effectiveness  model  developed 
by  Lie  (16)  is  sensitive  to  changing  operator  performance  expressions 
which  are  utilized  in  the  simulation. 

It  also  was  shown  that  the  overall  mean  performance  value  of  the  LFL 
curve  can  be  utilized  to  estimate  an  average  System  Effectiveness  value 
when  a large  number  of  missions  are  to  be  considered.  But  it  was  also 
demonstrated  that  the  LFL  curve's  mean  performance  value  could  not  be  used 
to  obtain  an  average  System  Effectiveness  value  if  only  a small  sample 
of  missions  was  to  be  studied.  In  this  type  of  situation,  the  average 
performance  had  to  be  obtained  directly  from  the  LFL  curve. 

4.3  Proposed  Future  Investigations 

Because  of  the  absence  of  any  significant  research  in  the  area  of 
operator  training/learning  and  their  subsequent  effect  on  operator  per- 
formance, the  field  is  open  to  any  number  of  studies  that  can  be  developed 
in  this  area. 

First  and  foremost,  a consistent  and  reliable  source  for  operator  per- 
formance data  should  be  identified.  Without  operator  performance  data,  the 
plotting  of  training/learning  curves  would  be  impossible,  and  if  the  curves 
can  not  be  obtained,  there  can  be  no  analytic  expression  for  operator  per- 
formance developed.  Gephart  and  Balachandran  (10)  suggested  that  human 


A 


68 

performance  data  could  be  obtained  from  the  training  programs  or  proficiency 
evaluations  of  the  operators  whose  performance  was  of  interest.  This 
suggestion  appears  to  be  logical  and  should  warrant  further  research  in  the 
areas  of  training  programs  and  proficiency  evaluations  to  ascertain  if 
they  would  constitute  a good  source  for  operator  performance  data.  Surely, 
there  are  other  sources  of  operator  performance  data  which  can  be  identified 
and  utilized,  and,  if  at  all  possible,  the  data  should  be  expressable  in 
terms  of  operator  performance  versus  time.  The  utilization  of  these  specific 
units  (performance  vs.  time)  would  facilitate  the  inclusion  of  the  operator 
performance  expression  into  all  the  System  Effectiveness  simulation  models. 

Lie's  simulation  model  is  rather  generalized  in  its  formulation.  The 
development  of  models  which  are  more  specific  in  their  formulation  and 
which  can  be  applied  in  detail  to  a particular  system  is  also  proposed  as 
a possible  future  investigation. 

Also,  in  Lie's  simulation  it  was  assumed  that  the  probability  of  mission 
success  due  to  environmental  conditions  was  a constant  in  each  environmental 
condition.  Furthermore,  the  probability  of  mission  success  due  to  the 
operator  was  assumed  to  be  independent  of  the  environmental  conditions. 

In  actuality,  the  performance  of  an  operator  is  almost  certain  to  be 
affected  by  the  environmental  conditions  in  which  the  operator  has  to  per- 
form. In  other  words,  an  operator  is  likely  to  perform  at  a higher  level 
in  good  weather  conditions  (moderate  temperatures,  low  humidity,  etc.)  than 
in  bad  weather  conditions  (high  or  low  temperatures,  mud,  snow,  etc.). 

The  dependence  of  operator  performance  on  the  environmental  conditions,  or 
the  conditional  probability  of  operator  performance  given  a certain  environ- 
ment, is  an  area  that  needs  to  be  researched. 
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Once  a source  of  operator  performance  is  identified,  then  analytical 
expressions  for  operator  performance  can  be  formulated.  Research  should  be 
conducted  in  the  area  of  applying  these  operator  performance  expressions  to 
the  various  other  Systems  Effectiveness  models  which  were  described  in 
references  (10)  and  (14).  The  sensitivity  of  these  models  to  various  ex- 
pressions for  operator  performance  should  also  be  studied. 
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Method  Utilized 
Integration 

Mission  Interval  Samples 


Average  Performance  (%)  Svste 
62 
60 


10th 

Interval 

59 

20th 

Interval 

54 

30th 

Interval 

55 

40th 

Interval 
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TABLE  4.1 

SUMMARY  OF  OPERATOR  PERFORMANCE  VALUES 
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00  10  1*1, MC 

1X1*1X1*10 

00  10  J=l,MM 

CALL  RANCUI  1X1 , IYI.YFLD 

CALL  OISTM.U  ,CP1  ,YpLl,KNl,NIl) 

ToMSI  I , J ) =Rn 1 
I X I*  I Y l 
1C  CONTINUE 

¥ih  I 7c  I 6 , 500 ) 

500  FORMAT  I 1 1 • , *T  ABLt  A.  TIME  INTERVAL  ESTrEcN  MISSION  STARTS’/) 
NR  I TE I 4 , 502 ) I I ,t*l  ,«CI 

502  FORMAT  I 1 • , 15x,’UNIT’/'  ' , 'MISSION'  ,131 12) 

DC  505  J* 1 , MM 

50  5 RR ITE lc,501)  J , I TiMSI I ,J)  , 1*1  ,MC) 

501  FORMAT ( ' ' , 17  , 10F12.2) 

...  COMPUTE  ACTUAL  MISSION  STARTING  T l.MES  ( CLOCK  TIMES) 


DO  20  1*1, MC 
CTNS IN*0 . 

DC  20  J* l , MM 

CTMSI  I , J ) *C TMS IN »TBMS ( ! ,J) 
CTMSIN*CTMS( I ,J) 

20  CONTINUE 
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52 

WRITSU.510) 

53 

510 

formati 1 i • , 'Table  a.  actual  mission  starting 

54 

WP.IT6I6.502M  1 ,I=l,MC) 

55 

00  515 

5„ 

515 

C • • • 

WRITE ( 6 ,501 ) U.tCTMSI I > J ) , 1=1 ,MC) 

C • • « 

L • • • 

GENERATE  ideal  mission  durations 

57 

1X2=21 

58 

DC  30  1=1, MC 

59 

1X2=1X2+20 

60 

DO  30  U= 1 i MM 

61 

CALL  SANCUt 1X2 , IY2 ,YFL2) 

62 

CALL  DISTM  X2  ,CF2.  YFL2.R.N2  ,N12) 

63 

D ( I , J ) =RN2 

64 

i X2= I Y2 

65 

30 

CONTI NUE 

66 

WRITE (6,520) 

67 

520 

FORMAT!  ' l'  , 'TABLE  C.  IDEAL  MISSION  CUF.ATIDNS' 

63 

WR  ITE (6, 50  2) ( I , I = 1,MC) 

69 

DC  52  5 J = I , M M 

70 

525 

• 

WR I Tc ( 6 , 50 1 ) J , ( 0 ( I , U ) , I = L >MC ) 

C • • • 

COMPUTE  IDEAL  MISSICN  FINISHING  TIM.ESICLQCX 

71 

DC  40  1=1, MC 

72 

00  -.0  J = t,.MM 

73 

CTMFI  1 , J)=CT.MS(  I , J)  +D<  I , J) 

74 

40 

CC NT  1 NuE 

75 

WR  t IE (6 ,530) 

76 

530 

FUPMAT ( *1’ , ’TABLE  0.  IDEAL  MISSION  FINISHING 
1) 

77 

WR  ITE I i,  502)  ( i , I =1  ,MC) 

78 

JC  535  J=l,MM 

79 

535 

WRITE  I 6, 501  ) JMCTMFi  I ,j)  , I = l,MC) 

C..  . 

C • • m 

GENERATE  MISSICN  TYPES 

SO 

IX  12= 121 

SI 

DC  1110  1*1,  MC 

82 

l X 12=  I X 12+  ICO 

53 

DO  1110  J*l,.MA 

84 

CALL  RANDUt IX 12 , IY 12 ,YFL12 1 

85 

CALL  DISTN(X12,CF12,YFL12,K.N12,M  121 

86 

TYPE!  I , J)*RM2 

67 

I X 12- I Y 12 

88 

1110 

CONTINUE 

39 

WRITE ( 6 , 1300) 

90 

1300 

FORMAT! • l*  , 1 TABLE  E.  MISSICN  TYPES1/) 

91 

WR  ITE (6, 502 ) ( I , 1 = 1 ,MC) 

92 

on  i3io  j*i,mn 

93 

1310 

WRITE (6,501)  J, ( TYPEl I , J ) , I * l » M C ) 

c.. 

c.. 

c.. 

c.. 

c.. 

cc. 

c.. 


94 


GENERATE  T I KC  INTERVAL  between  failures  and  duration  cf  repair 
COMP JT  = ACTUAL  FAILURE  STARTING  TIMES  AND  ACTUAL  REPAIR  FINISHING 
T I PCS  I CLOCK  TIMES) 

FOR  THE  FAILURE  INDUCED  TY  HARDWARES 
WR I TE I 6 i 539  ) 
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95  539  FORMA  7 ( ' 1 ' , 1 THE  FCLLCWlNG  TABLES  SHOW  '/'  '.‘TIME  INTERVAL  3ETWEEN 

1 FAILURES!  TErtF  ) , 1 / ' 'ACTUAL  FAILURE  STARTING  T I RES (CTHFS ) , ' / • 
2'OURATICN  CF  HcPHP.(KTH),  '/•  • , <ANJ  ACTUAL  «EPAU  FINISHING  TIMES! 
3CTflRF  ) ' / ' 1 , • r OR  THE  FAILURE  InCUCEC  aY  HAKCWASES'J 


96 

1X3=31 

97 

I X4=4l 

93 

9C  540  1*1, MC 

99 

WRITS (£,541) I , I 

100 

541 

FORMAT! '1* , 'TABLE  F . ' , 12, 1 .HARDWARE  1N0JC 

EO',13//’  *,•  failure  amj 

I/'  ', 'REPAIR  INCEX* ,6X, • TdhF' , 7X, 'CTHFS' , 

5X  , 1 R T m ' ,7X  ,'CTHRF'l 

101 

cthfsz=o. 

IC2 

R THZ  *0 • 

103 

1X3= I X3+100 

1 04 

1X4=1X4*100 

105 

J=  1 

106 

545 

CALL  RSFAIL ( 1X3, 1X4, IY3, 1 Y4,X3,Cr3,M  3,X4 
lTBhFj  ,RTHO,CTHFSC,CTHP.Fj,CTHrSZ,ATHZ) 

, C F 4 , N 1 4 , 

107 

TBmF ( I ,J)=TttHFO 

103 

CTHFS!  I , J ) = C T HFS  0 

109 

P.TH  1 I , J ) =R  THQ 

1 10 

CTHP.F ( I ,J)=CTHRFO 

111 

JH( 1 ) =J 

1 12 

WRITEI6,542)J , TBHF ( ! , J ) , CTHFS! 1 , J ) ,RTH(  I, 

J)  , CTFRF ( I, J) 

l 13 

CThFSZ  = CThfS! I ,J  ) 

1 14 

RThZ=RTH( I , J) 

115 

IFtCTHFSZ.GE.CTMF! I ,MM) ) GO  TO  540 

l 16 

J*  J * l 

117 

i X3= I Y3 

1 18 

I X 4=  1 Y4 

1 19 

GO  T „ 545 

120 

540 

CONTINUE 

121 

542 

r 

FORMaT (•  • , I12.4F12.2J 

• • • 

C C • « • 

c... 

F CR  THE  FAILURE  INCUCEO  BY  OPERATORS 

122 

rtRITE  16,550 

123 

550 

FORMAT!  ' 1'  , 'THE  FOLLOWING  TABLES  SHOW/' 

','TIFE  INTERVAL  EETw  EE.N 

1FAILUP.  £S!T  6Cf)  , ' / 1 '.'ACTUAL  FAILURE  STAR 

tlNU  TIMESiCTC.Fi)  , •/•  • , 

2DURAT I ON  CF  REPAIR 1RTC) ,'/ ' ','  A,  *C  . ACTUAL 

REPAIR  FINISHING  TIMES!' 

3TCRFJ'/'  ' , ' F'JR  THE  FAILURE  INDUCES  GY  C’EP.AT  CF  S » I 

124 

I X5=5  l 

125 

1 xa=6  l 

126 

DC  5d 1 1=1, MC 

127 

WP ITE ( a,  552  ) I , I 

l 23 

552 

FORMA  It  ' l ', 'TABLE  G.  ' . I 2 > ' . GP£R  ATCR  INDUCED • , I III  ' ' 

, 'FAILURE 

1/ ' ', 'REPAIR  INDEX ' ,3X, 'T3CF ', 7X, 'CIOFS ', Sx, ’R TC  ,7X 

, ' CTCXF ' ) 

125 

C TCF  5Z=0 . 

130 

RTCZ»0. 

131 

1X5=1X5*100 

1 32 

1X6  = 1 X6*  100 

133 

J«  l * J H ! I ) 

l 34 

555 

CALL  REFAIL (1X5, 1X6, (Y5, I Y6 , X5 , CF 5, N I 5, Xo , CF6 ,N 16 , 
1T30FC ,RTOD,CTOFSG,CTORFO,CTOFSZ ,RTOZ) 

135 

TohF!  I ,J  ) = TBOFO 

136 

lTHFS! I , J ) =CTCFSO 

137 

i’.TH(  I , J)=RTOC 

138 

CTHRF ( 1 , J ) =CT  CRFO 

139 

JO!  I ) =J 

140 

RP.  ITE(6, 54  2JJ  ,TBHF!  I , J ) , CThFS!  I , J ) »KTH(  I • J ! ,CTHRF(  I , 

J) 

141 

CTCFSZ  = CThFS(  ! , J) 
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142 

RTG2=RTH< I , J) 

143 

IF (CTCFS2. GE.CTMFl I.MMJ1  GC  TO  551 

144 

J*J+i 

145 

I X 5=  [ Y 5 

146 

IX6*[  Y6 

147 

GC  TO  555 

148 

551 

c.. . 

CONTINUE 

C . . • 

REARRANGE  FAILURE  STARTING  TIMES  AND 

C • • • 

£ 

REPAIR  FINISHING  TIMES  IN  ASCENDING  OROGR 

149 

00  1000  1=1, MC 

150 

KA=J3( I )-l 

151 

OC  1000  11*1, KA 

152 

JASJOI I ) - l 

153 

00  1000  K=  l , J A 

l 54 

1 F 1 C ThFSI I ,K+1 1 .GE.CTrtrSt  I ,K)  ) GO  TC  1001 

155 

TEM^CThFSI  I , K ) 

156 

CT nr S I 1 ,K) =CTHFS  l l ,.<H  > 

l 57 

C ThcS ( I,<»l)  = TE.MP 

: 53 

1001 

IFtCTi-PFf  I ,X«-l  I.GE.CTHRFI  I.K)  ) GO  TC  1000 

l 5S 

TcM01=CTHRF(1 ,X) 

160 

CTMRFI  I , X ) =CT PRF 1 1,01) 

lei 

CTHRF  1 1 , R+  l ) = T EMF  l 

lo2 

1000 

CONTINUE 

c • , • 

r 

performance  of  unit  in  different  environment 

163 

U • • • 

90 

1X11*111 

l 64 

OC  100  1=1, MC 

165 

1X11=1X11+100 

1 66 

OC  100  J=l,MM 

167 

CALL  RANCUf !Xl 1,  1Y1 1 , YFLil  ) 

168 

CALL  OtSTNlXU.CHi.YFLli  ,RM1,M  11) 

169 

ENVMT  I I , J) =RN 1 1 

1 70 

! X 1 1 = I Y 1 1 

1 71 

100 

CONTINUE 

172 

RR1T=16,530> 

l 73 

530 

FORMAT! ’ 1* , 'TABLE  H.  PERFORMANCE  CF  UNIT  FOR 

174 

WRITE  16. 502 ) I I , 1 = 1 ,«C) 

175 

00  536  0*1, MM 

176 

5 3o 

C... 

WR  IT;  16, 50  1 ) J , ( ENVMTI  I ,J ) ,1*1  , MC) 

C... 

OPERATOR  PERFORMANCE  CURING  MISSION 

177 

OC  300  1 = 1 ,MC 

178 

TP  1*0. 

l 79 

t r>  f*  r ? m 

l 80 

00  300  J*  1 , M.M 

l 31 

IF  ICTMFI  I , J) .LE. TPF)  GC  TO  310 

1 32 

CS?2=CT  MS  1 I , J )— T ? I 

l 83 

CF=<2  = CT»F(  I ,J)  -TPF 

184 

T P T=  T P F-  TP  I 

195 

GPU  , J)*Yl  I I ) MYl  l 1 )/TPT*Y2I  t >/  El  l ) »t  i.*cXPI 

ItXPI— 3m*CFP2l-6XP(-9(I)*TPm  )/0(  I,J) 

r?  i=tpf 

TPf  *T  P 1 ♦ TP  I I ) 

GO  TO  300 

I CSP*CT;'S(  I , J J -TP  I 
CFP*CTMC(  I , JI-TPl 
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191  OP  1 1 . J)=Yl  l I)  *-Y2t  I )/  1 3 ( I ) *01  l , J) ) *1  ExPI-31  I ) *C SP) -E XP t-BI I 1 *CFPI) 

192  3 OC  CONTINUE 

t93  WRITEI6.620) 

194  620  FORMAT  I • l' , 'TABLE  I.  OPERATOR  PERFORMANCE  DURING  NISSICN'/J 

195  WRITElo, 591)1 I .1*1 ,MCl 

196  591  FUftMAT  I 1 *,  11X  , 'OPERATOR  •/ » ' i '“l  SS  l ON  • , 1 0U2  ) 

1 97  OC  625  J® 1 . MM 

193  625  WRITE  It, 501)  J,(0P(I.J1  .l«l.MCl 
C • • • 

C...  COMPUTE  EXPECTEO  MISSION  DURATIONS 

c • • • 

199  OC  1100  l=l,MC 

2 00  00  LI  00  J* 1 »MM 

201  1100  3EQI 1 ,J)*0( I .Ji/(ENVMT(I ,J)*OP(  l, Jll 

202  WRITE (6, 15J0) 

203  1500  FORMAT! 1 1' TABLE  J.  EXPECTED  MISSION  DURATIONS'/) 

204  «P.ITE16, 502)1  1. 1 = 1, MC) 

205  JC  1550  0=1, MM 

206  1550  WRITE  16,501)  J , I DEG  1 I , J ) , I •=  l , MC  1 
C... 

C...  COMPUTE  EXPECTED  MISSION  FINISHING  TIMES 
C # • • 

207  UC  1200  1=1, MC 

2CS  DO  1200  J=1,MH 

209  IFICSGt  l , J I .L6.TYP5I  I , J) ) GO  TO  1210 

2 10  CTmF  l I , J)  = CT.-I5l  I , J)  ♦TYPE!  I,  J) 

211  GO  TO  1200 

212  1210  CTMrl  I , J )=CTMS(  I , J ) *-DECl  I , J) 

2 13  12  GC  CONTINUE 

2 14  WRITfc  ( t , 1600 

215  1600  FORMAT  I ' 1'  , ' T A3U  E X.  EXPECTED  MISSION  FINISHING  TIMES'/) 

2 16  WRITE  I 6,502 ) I I , I = l , MC ) 

217  DU  lo  50  J=  1 , MM 

218  1650  WRITE (6,501  ) J, I C7HF I I , J ) , l«l ,MC) 

w • • • 

C...  COMPUTE  MISSION  RELIABILITY  AND  AVAILAoILlTY 

C • • • 

CC...  INITIALIZATION 

c«,  • 

219  OC  l 10  1 = 1,  MC 

220  CThRF  II  I )*0. 

221  KHl(l)«l 

222  110  CONTINUE 

223  DO  200  J« 1 , MM 

224  DO  200  X*l , MC 

225  19  76  IF  ICTMSU  , J)  .1  E. CTHRF  IX,KH1 IX)  ) 1 GC  TO  210 

226  CTHRF  1 1 K ) *CT.iRF  I K , XHl  I X) ) 

227  XH  1 1 K ) ■ KH 1 1 X 1 *1 

223  GO  TO  1976 

229  210  IF ( ICTMS IX , J) . Gc .CThRF 1 1 X ) I .AND.ICTMS IX, J ) .LT. 

ICTHFSIX.XHIIKI) ) ) GO  TC  220 

230  Ah(K,J)=0. 

2 31  RH I X, J ) *0. 

232  P.H  A I x , J ) =0  • 

233  50  TO  200 

234  2 20  AH  I K , J ) = 1 • 

235  IF  I (CTMF I X , J)  .GT  .CTHRF MX)) .AND.  I CTMF (X,J1 .LE. 
lCThFSIK.KHl IX) ) ) ) GO  TO  230 

240  RH(X,J)=0. 


226 
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2 37  AHA<K.J>=CTFFSIK,KHUK)  )-CTMSlK,J) 

23b  GO  TJ  200 

239  2j0  IF  iGEOlK.J)  .LE .TYPE ( K , J ) ) GC  TO  1120 

290  Rm(K,J)=0. 

291  RHAK,  J)=TYPEIK,  J) 

292  GO  TO  200 

293  1120  RHK,J>=1. 

299  P.IIAfK,  J)»0EC(K,J) 

295  200  CONTINUE 

296  WRITE!6,535) 

297  565  FORMAT!  *1'  ,'TASLE  L.  AV4LA3ILITY  Cr  EACn  UNIT'/I 

293  WRITE(b,581)< l,l=i,MC) 

249  531  FORMAT!'  ’ , 1 5 X , 'UNIT  •/•  1 , 'MISS  ION  ' ,10112) 

250  30  533  J*1,NM 

251  533  aRITSU.SJI)  J , I AHU  , J I , I * 1 ,MC  I 

252  wR!Tfcfa,600) 

253  600  FORMAT!  1 l • , 'TABLE  «.  MISSION  REL I Adi  LI  TY  GF  UMT  •/ 

ITS : IMPLICATION  OF  (A)  '/'  ',15X,'A  IS  THE  CJrUTICN  CF 

2100  C AAR  I EG  Out  BY  UNIT'/! 


2 54 

»e  1TE ( 6, 561 II 1 , I = 1,MC) 

25s 

DC  605  J=l,MM 

2 So 

60  5 

WR1 TE  < 6 ,602)  J , ( RM  1 , J)  ,P.HA<  1 , J ) , 1 = 1 , MC  ) 

257 

o 02 

FORMAT ( • < , 17  , 10  < Fi.G, > ( 1 ,F7.2,  * ) •) ) 

C • • • 

C • • • 

COMPUTE  MISSION  EFFECT  IVENESS 

25S 

OC  90  0 1 = 1 ,MC 

259 

SUMC  1 = 3. 

260 

DO  4 10  J = l,  MM 

261 

EMI  1 , Ji =Ah ( I , J ) * RH  l I , J ) 

262 

SuMC I = SuMC i +EM  ( I , J ) 

263 

410 

continue 

264 

EMC! I ) = SUMC I / MM 

2 o5 

90  0 

CONTINUE 

266 

..RITE  16,630) 

2 67 

630 

FORMAT!  ' l'  , 'TABLE  N.  MISSION  EFFECTI  VENESS  OF  EACH 
1MISSION'/) 

UN  i 

263 

wRI  TE (e  ,502) ( I , I =1 ,MC) 

269 

DO  635  J=I  ,MM 

270 

6 3 5 

UK  I TE ( 6 , 50  l)  Jr ( EMI  1 , J ) , 1 = 1 ,MC) 

271 

WR  I TE  ( 6,650)  ( I ,1  = 1 ,MC) 

2 72 

650 

FORMAT ('  1' , 'TABLE  C.  OVERALL  MISSION  EFFECTIVENESS 

1 1 • //•  ' ,7X , 101 12  ) 

CF 

273 

UR  ITE  ( 6,651 ) ( EMC  (I  ) , 1 = 1 ,,MC> 

274 

651 

FORMAT ( 1 • ,7X,10F12.2) 

275 

WRITE ( 6,5000) 

2 76 

5000 

FORMAT! ’ l ' ) 

2 77 

STOP 

2 78 

C... 

END 

C..  . 

c... 

GENERATE  RANDOM  NJ.M9ERS 

2 79 

SUBROUTINE  RANCU!  IX.IY.YFLl 

280 

IY* [<*65539 

281 

IF!  IYI  5,6,6 

28  2 

5 

I Y=  I Y *2  I47483647*l 

283 

6 

YFL=  I Y 

2 84 

YFL=YFL».46s66I3E-9 

285 

RETURN 

' < , 10  X , • NO 
M I S i 1 0 N PER 


T FOR  EACH 


UNIT  t M £ 1 1 ) 
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236  END 

c... 

C...  FIT  RANCCM  NJM2EKS  INTO  0 1ST.1?  1 3UT  ION 


2 37 
2 S3 
2 39 
2 90 

291 

292 

293 
29-. 

295 

296 
257 

298 

299 

300 


SUBROUTINE  C1STN(X,CF,RV,RN,NI) 

DIMENSION  XI Ml  ,CFIM) 

KV=IJJ.»RV 

IF  lHV.LE.CfU)  ).  GC  TO  20 
I *2 

90  J = 1-1 

IF (IRV.GT.CFI J) ) .ANO.IRV.LE.CFI  l) })  30  TO  30 
1=1  + 1 
30  TO  90 
20  RN=xm 
■SC  TO  100 
30  3N=X ( I ) 

100  RETURN 
EMC 

C . . « 

C...  GENERATE  TIME  INTERVAL  BETWEEN  FAILURES  AND  UURAT1GN  CF  REPAIR  FOR 
C...  HARDWARE  AND  OPERATOR  INDUCED  FAILURES. 

C...  COMPUTE  ACTUAL  FAILURE  STARTING  TIMES  AND  ACTUAL  REPAIR  FINISHING 
C...  TIME  SI  CLOCK  TIMES) 

C... 


301 

302 
3C3 

309 

305 

306 

307 
3C3 
3 09 

310 

311 

312 


SUBROUTINE  P.EFAIL  I IXSl.  IXS2,  IYS1,  I YS2 , X S L , Cf S l , N I S 1 , X S2  ,Cr  S2 . N IS2  , 
lTMSr,RTS,CTSFS,CTSRF,CTSFSI,RTSIN) 

01  PENSION  XSI  INI S l)  , CFS  II IS  1 1 , XS21NI S2) .CFS2 I NIS2J 

CALL  RANDUI IXS1 . I YSl rYFLSI I 

CALL  GISTN ( XSI ,CF SI ,YF LSI ,RNSl,M  Si) 

T 2Sr  = KN  $ l 

CALL  RANDUl 1XS2. 1 YS2 ,YFLS2) 

CALL  0 ISTN  I.'.S2#CFS2iYFLS2iRfiS2tM$2) 

RT  S = RNS2 

CTSFS=CTSFSI*T3SF+RTSIN 

CTSRF=CTSFS+RTS 

RETURN 

End 


SENTRY 


